Introduction

Space Robotics Bench (SRB) is a comprehensive collection of environments and tasks for robotics research in the challenging domain of space. It provides a unified framework for developing and validating autonomous systems under diverse extraterrestrial scenarios. At the same time, its design is flexible and extensible to accommodate a variety of development workflows and research directions beyond Earth.

Key Features

Highly Parallelized Simulation via NVIDIA Isaac Sim: SRB supports thousands of parallel simulation instances to accelerate workflows such as online learning, synthetic dataset generation, parameter tuning, and validation.
On-Demand Procedural Generation with SimForge: Automated procedural generation of simulation assets is leveraged to provide a unique scenario for each simulation instance, with the ultimate goal of developing autonomous systems that are both robust and adaptable to the unpredictable domain of space.
Extensive Domain Randomization: All simulation instances can be further randomized to enhance the generalization of autonomous agents towards variable environment dynamics, visual appearance, illumination conditions, as well as sensor and actuation noise.
Compatibility with Gymnasium API: All tasks are compatible with a standardized API to ensure seamless integration with a broad ecosystem of libraries and frameworks for robot learning research.
Seamless Interface with ROS 2 & Space ROS: Simulation states, sensory outputs and actions of autonomous systems are available through ROS 2 middleware interface, enabling direct interoperability with the vast (Space) ROS ecosystem.
Abstract Architecture: The architecture of SRB is designed to be modular and extensible, allowing for easy integration of new assets, robots, tasks and workflows.

📑 If you have any questions or suggestions regarding this documentation, don’t hesitate to reach out to us! More often than not, a lack of understanding is a result of poor documentation… Therefore, we are always looking to improve it!

🚀 New to SRB? Start with the System Requirements, follow the Installation guide, then work through Basic Usage.

Environments

The Space Robotics Bench provides a variety of environments that simulate diverse scenarios in the challenging domain of space:

Manipulation - Stationary manipulation skills and sequences in various contexts
Mobile Robotics - Traversal and navigation under extraterrestrial conditions
Mobile Manipulation - Complex combination of mobility and manipulation
Digital Twins - High-fidelity digital twin environments modelling real-world facilities

Environments – Manipulation (Static)

The Space Robotics Bench provides a set of manipulation environments for fixed-base robot arms.

Manipulation (`_manipulation`)

Template environment for fixed-base manipulation tasks. Configure via env.robot=<manipulator>.

_manipulation


Episode Length	10.0s (500 steps)
Reward Terms	4
Default Robot	Franka

Compatible Robots

Canadarm3 | Franka | Kinova J2N6S | Kinova J2N7S | Kinova Gen3N7 | Levion Arms | So Arm100 D5 | So Arm100 D7 | Unitree Z1 | Ur3 | Ur3E | Ur5 | Ur5E | Ur10 | Ur10E | Ur16E | Ur20 | Ur30 | Random Ur Manipulator | Vispa | Zeroglab Arms

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.1	–
penalty_joint_torque	Linear	-0.000025	–
penalty_joint_acceleration	Linear	-0.0005	–
penalty_undesired_robot_contacts	Threshold	-0.5	–

Debris Capture (`debris_capture`)

Capture and securement of floating debris in microgravity with a fixed-base manipulator


Episode Length	10.0s (500 steps)
Reward Terms	11
Default Robot	Franka

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.1	–
penalty_joint_torque	Linear	-0.000025	–
penalty_joint_acceleration	Linear	-0.0005	–
penalty_undesired_robot_contacts	Threshold	-0.5	–
reward_distance_ee_to_obj	Gaussian	16	0.08
reward_grasp	SoftThreshold	16	–
penalty_debris_velocity_lin	Linear	-3	–
penalty_debris_velocity_ang	Linear	-1	–
reward_approach_velocity_alignment	Hyperbolic	4	0.1
penalty_approach_speed	Linear	-2	–
penalty_debris_too_far	Linear	-8	–

Termination semantics. distance_debris_too_far drives the penalty_debris_too_far reward and does not set terminated; the finite horizon is reported as truncated.

Excavation (`excavation`)

Extraction of granular material (regolith)


Episode Length	8.0s (400 steps)
Reward Terms	13
Default Robot	kinova_gen3n7

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.05	–
penalty_joint_torque	Linear	-0.00001	–
penalty_joint_acceleration	Linear	0	–
penalty_undesired_robot_contacts	Linear	0	–
reward_approach_pile	Gaussian	1.5	0.15
reward_scoop_depth	Tukey	2	–
reward_scoop_orientation	GaussianTracking	1	0.26
reward_lift	Hyperbolic	2	0.2
penalty_jerk	Linear	-0.02	–
penalty_joint_acc	Linear	-0.000001	–
penalty_undesired_contacts	Linear	-0.5	–
terminal_particle_count_scaled	Scaled	10	–
terminal_success_bonus	Scaled	50	–

Note: The observed heightmap is derived at runtime by binning the live regolith-particle positions onto the observation grid (PhysX particle sets are not raycastable), so it tracks the pile as it is reshaped. The pedestal IMU (imu_pedestal) is only active with env.num_envs=1; under parallel environments it is disabled (a warning is logged) and its observations read zeros.

Excavation Demo (`excavation_demo`)

Manual excavation demo with presentation-oriented lunar regolith defaults

excavation_demo


Episode Length	1800.0s (90000 steps)
Reward Terms	13
Default Robot	kinova_gen3n7

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.05	–
penalty_joint_torque	Linear	-0.00001	–
penalty_joint_acceleration	Linear	0	–
penalty_undesired_robot_contacts	Linear	0	–
reward_approach_pile	Gaussian	1.5	0.15
reward_scoop_depth	Tukey	2	–
reward_scoop_orientation	GaussianTracking	1	0.26
reward_lift	Hyperbolic	2	0.2
penalty_jerk	Linear	-0.02	–
penalty_joint_acc	Linear	-0.000001	–
penalty_undesired_contacts	Linear	-0.5	–
terminal_particle_count_scaled	Scaled	10	–
terminal_success_bonus	Scaled	50	–

Modular Assembly (`modular_assembly`)

Pick-and-place modular elements into an assembly configuration

modular_assembly


Episode Length	20.0s (1000 steps)
Reward Terms	10
Default Robot	Franka

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.1	–
penalty_joint_torque	Linear	0	–
penalty_joint_acceleration	Linear	-0.0005	–
penalty_undesired_robot_contacts	Threshold	-32	–
reward_top_down_orientation	GaussianTracking	16	0.25
reward_pair_position	Linear	8	–
reward_pair_orientation	Linear	2	–
reward_peg_hole_entrance_alignment	Linear	8	–
reward_peg_hole_alignment	Linear	64	–
reward_success	Linear	512	–

Multi Peg In Hole (`multi_peg_in_hole`)

A variation of peg-in-hole assembly with multiple pegs and holes


Episode Length	40.0s (2000 steps)
Reward Terms	19
Default Robot	Franka

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.1	–
penalty_joint_torque	Linear	-0.000025	–
penalty_joint_acceleration	Linear	-0.0005	–
penalty_undesired_robot_contacts	Threshold	-0.5	–
reward_top_down_orientation	GaussianTracking	1	0.25
reward_distance_ee_to_obj	Hyperbolic	0	0.25
reward_grasp	Linear	8	–
reward_lift	Linear	4	–
reward_align_peg_to_hole_primary	GaussianTracking	0	0.4
reward_align_peg_to_hole_secondary	Gaussian	0	0.2
reward_distance_peg_to_hole_entrance_gradual	Hyperbolic	0	0.16
reward_distance_peg_to_hole_entrance	Gaussian	0	0.04
reward_distance_peg_to_hole_bottom	Gaussian	0	0.003
reward_distance_ee_to_objs	Linear	2.5	–
reward_align_pegs_to_holes_primary	Linear	8	–
reward_align_pegs_to_holes_secondary	Linear	4	–
reward_distance_pegs_to_holes_entrance_gradual	Linear	8	–
reward_distance_pegs_to_holes_entrance	Linear	32	–
reward_distance_pegs_to_holes_bottom	Linear	256	–

Multi Sample Collection (`multi_sample_collection`)

A variation of sample collection with multiple samples


Episode Length	22.5s (1125 steps)
Reward Terms	12
Default Robot	Franka

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.1	–
penalty_joint_torque	Linear	-0.000025	–
penalty_joint_acceleration	Linear	-0.0005	–
penalty_undesired_robot_contacts	Threshold	-0.5	–
reward_top_down_orientation	GaussianTracking	1	0.25
reward_distance_ee_to_obj	Gaussian	0	0.15
reward_grasp	Linear	4	–
reward_lift	Linear	8	–
reward_distance_obj_to_target	Gaussian	0	0.15
penalty_obj_fallen	Linear	0	–
reward_distance_ee_to_objs	Linear	2.5	–
reward_distance_objs_to_target	Linear	32	–

Peg In Hole (`peg_in_hole`)

Precision assembly task requiring the robot to insert a peg into a hole


Episode Length	10.0s (500 steps)
Reward Terms	13
Default Robot	Franka
Multi Variant	`multi_peg_in_hole`

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.1	–
penalty_joint_torque	Linear	-0.000025	–
penalty_joint_acceleration	Linear	-0.0005	–
penalty_undesired_robot_contacts	Threshold	-0.5	–
reward_top_down_orientation	GaussianTracking	1	0.25
reward_distance_ee_to_obj	Hyperbolic	4	0.25
reward_grasp	SoftThreshold	8	–
reward_lift	GaussianZoneTracking	4	0.08
reward_align_peg_to_hole_primary	GaussianTracking	8	0.4
reward_align_peg_to_hole_secondary	Gaussian	4	0.2
reward_distance_peg_to_hole_entrance_gradual	Hyperbolic	8	0.16
reward_distance_peg_to_hole_entrance	Gaussian	32	0.04
reward_distance_peg_to_hole_bottom	Gaussian	256	0.003

Sample Collection (`sample_collection`)

Grasping and collection of samples on planetary surfaces


Episode Length	7.5s (375 steps)
Reward Terms	10
Default Robot	Franka
Multi Variant	`multi_sample_collection`

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.1	–
penalty_joint_torque	Linear	-0.000025	–
penalty_joint_acceleration	Linear	-0.0005	–
penalty_undesired_robot_contacts	Threshold	-0.5	–
reward_top_down_orientation	GaussianTracking	1	0.25
reward_distance_ee_to_obj	Gaussian	4	0.15
reward_grasp	SoftThreshold	6	–
reward_lift	GaussianZoneTracking	12	0.15
reward_distance_obj_to_target	Gaussian	32	0.15
penalty_obj_fallen	Linear	-16	–

Termination semantics. obj_floor_z drives the penalty_obj_fallen reward. A fallen sample does not set terminated; the finite horizon is reported as truncated.

Screwdriving (`screwdriving`)

Fastening of a pre-inserted bolt


Episode Length	15.0s (750 steps)
Reward Terms	13
Default Robot	Franka

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.1	–
penalty_joint_torque	Linear	-0.000025	–
penalty_joint_acceleration	Linear	-0.0005	–
penalty_undesired_robot_contacts	Threshold	-0.5	–
reward_top_down_orientation	GaussianTracking	8	0.04
penalty_object_top_down_orientation	Linear	-64	–
reward_distance_ee_to_bolt_driver_slot	Hyperbolic	12	0.4
reward_distance_ee_to_bolt_driver_slot_close	Hyperbolic	32	0.02
reward_distance_ee_to_bolt_driver_slot_precision	Hyperbolic	512	0.005
reward_contact	SoftThreshold	4	–
reward_screwing	Scaled	128	–
penalty_distance_obj_to_target_too_far	Threshold	-256	–
reward_distance_obj_to_target	Linear	2048	–

Solar Panel Assembly (`solar_panel_assembly`)

Complex assembly sequence involving solar panel components


Episode Length	50.0s (2500 steps)
Reward Terms	17
Default Robot	Franka

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.1	–
penalty_joint_torque	Linear	-0.000025	–
penalty_joint_acceleration	Linear	-0.0005	–
penalty_undesired_robot_contacts	Threshold	-0.5	–
reward_top_down_orientation	GaussianTracking	1	0.25
reward_distance_ee_to_objs	Linear	2.5	–
reward_grasp	SoftThreshold	8	–
reward_lift	Linear	4	–
reward_align_pegs_to_holes_primary	Linear	8	–
reward_align_pegs_to_holes_secondary	Linear	4	–
reward_distance_pegs_to_holes_entrance_gradual	Linear	8	–
reward_distance_pegs_to_holes_entrance	Linear	32	–
reward_distance_pegs_to_holes_bottom	Linear	256	–
reward_distance_ee_to_panel	Hyperbolic	10	0.25
reward_panel_lift	GaussianZoneTracking	16	0.1
reward_distance_panel_to_target_gradual	Hyperbolic	32	0.1
reward_distance_panel_to_target	Gaussian	1024	0.015

Environments – Mobile Robotics

Environments for mobile robot navigation and locomotion tasks.

Aerial (`_aerial`)

Template environment for aerial mobile robots. Configure via env.robot=<aerial_robot>.

_aerial _aerial


Episode Length	10.0s (250 steps)
Reward Terms	1
Default Robot	Ingenuity

Compatible Robots

Crazyflie | Ingenuity

Reward Breakdown

Term	Type	Weight	Scale	Notes
penalty_action_rate	Linear	-0.1	–

Ground (`_ground`)

Template environment for ground mobile robots. Configure via env.robot=<ground_robot>.

_ground _ground


Episode Length	10.0s (250 steps)
Reward Terms	1
Default Robot	Cadre

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale	Notes
penalty_action_rate	Linear	-0.1	–

Orbital (`_orbital`)

Template environment for orbital mobile robots. Configure via env.robot=<orbital_robot>.

_orbital


Episode Length	10.0s (250 steps)
Reward Terms	2
Default Robot	Cubesat

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale	Notes
penalty_action_rate	Linear	-0.1	–
penalty_fuel_consumption	Linear	-1	–

Aerial navigation to a dynamic waypoint with terrain-clearance tracking


Episode Length	60.0s (1500 steps)
Reward Terms	7
Default Robot	Ingenuity

Compatible Robots

Crazyflie | Ingenuity

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.1	–
reward_altitude_tracking	HyperbolicTracking	4	1
reward_waypoint_approach	Hyperbolic	16	2
penalty_tilt_angle	OffsetLinear	-2	–
reward_terrain_clearance	HyperbolicBarrier	8	0.5
penalty_energy	Linear	-0.5	–
reward_waypoint_reached	Threshold	32	–

Beam Transport (`beam_transport`)

Two-robot collaborative transport of a rigid beam payload to a target point

beam_transport

Beam transport is the second centralized-policy multi-robot surface task (the companion task is formation following). Two Spot quadrupeds carry a single rigid beam, pre-mounted across the pair, to a target point on procedural lunar terrain (MoonSurface). A single shared policy drives both robots, with their observations concatenated in a fixed order and one joint action split across the pair (the same centralized formulation as formation following).

Payload interface. By default (beam_kinematic_mount_enabled = true) the beam is kinematically mounted to the two-robot support frame: its pose follows the midpoint and heading of the robot pair rather than resting on grasp or contact dynamics. This deliberately sidesteps grasp and contact-stability physics, so the task scores coordinated transport-frame motion — even a solved policy under this mount would not yet demonstrate contact-mediated cooperative carrying, which a more physically detailed coupling (future work) would add. The reward still enforces support-slot alignment and stable, level transport. A free-beam mode (beam_kinematic_mount_enabled = false) drops the mount for ablations.

Benchmark protocol (intended; no trained policy ships yet — this is a draft). The algorithm of record is DreamerV3 (hyperparams/dreamerv3.yaml); validation specs live under hyperparams/validation/. A policy must be trained first:

# Train (DreamerV3 is the algorithm of record)
srb agent train --env beam_transport

# Evaluate a trained checkpoint (reports the metrics below)
srb agent eval --algo dreamer --env beam_transport --model <path/to/checkpoint>

The reference runs used 64 parallel environments for 50M agent steps at a 25 Hz control rate; episodes are 30 s (750 steps). Train over independent random seeds and report mean ± standard deviation.

Evaluation. Each rollout reports the closest beam-to-target distance reached (min_beam_distance_to_target, m) and the success-hold streak, alongside success_rate; expect success_rate = 0 until the transport is solved, with the minimum distance as the primary partial-competence signal.


Episode Length	30.0s (750 steps)
Reward Terms	15
Default Robot	Spot

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.1	–
reward_beam_position	Gaussian	16	1.5
reward_beam_midpoint_alignment	Gaussian	10	0.25
reward_robot_support_slots	Gaussian	12	0.2
reward_beam_progress	Scaled	8	–
reward_attachment_balance	Gaussian	4	0.15
reward_success	Linear	24	–
penalty_beam_tilt	Linear	-4	–
penalty_beam_drop	Linear	-16	–
penalty_beam_speed	Linear	-2	–
penalty_joint_torque	Linear	-0.000025	–
penalty_joint_acceleration	Linear	-0	–
penalty_undesired_lin_vel_z	Linear	-0.5	–
penalty_undesired_ang_vel_xy	Linear	-0.1	–
penalty_gravity_rotation_alignment	Linear	-2	–

Success criterion. A rollout succeeds once the beam holds the target pose for success_hold_steps = 8 consecutive control steps (0.32 s) under all of: planar distance to target ≤ target_zone_radius_xy = 0.20 m; beam-to-midpoint and support-slot alignment within beam_attachment_radius_xy = 0.30 m; planar speed ≤ target_speed_threshold_xy = 0.35 m/s; and an uprightness gap 1 - |cos θ| ≤ beam_tilt_tolerance = 0.20 (θ ≲ 37° from vertical). The episode is finite-horizon (30 s). Each reset randomizes both Spots within a ±1.5 m planar box with full yaw, so the pair must re-establish the carry from varied starts.

Observation & action spaces. The policy observes, per robot, planar position and velocity and the offset to the beam, plus the beam pose (position and a 6D rotation), the beam-to-target offset and distance, an assignment-invariant support-slot error, and the beam’s planar and angular velocity together with the robot-pair midpoint. The action drives both Spot quadrupeds.

Status and failure signature. Validation rollouts of the centralized-policy baselines reach zero task success, but the failure shape differs from formation following: an intermediate checkpoint approached the 0.20 m distance gate to within a few millimetres (without ever entering it), then longer training regressed to well outside it and never re-approached. That near-solution-then-regression points to optimization and reward-stability issues rather than a model-capacity ceiling. The task is a validated benchmark and workflow contribution — stable training, logging, and reproducibility — with solving the transport left open.

Sim-to-real path. A unit-tested per-robot velocity command interface and the shared SRB domain-randomization terms (terrain, friction, delays, command drift) form a staged transfer plan; no hardware or ROS deployment result is claimed.

Note — the Reward Breakdown table above is regenerated by srb doc docgen and reflects the core transport-shaping terms. The task additionally applies per-robot locomotion-stability penalties (joint torque and acceleration, vertical velocity, body roll/pitch rate, and projected-gravity alignment) that steady the carry; rerun srb doc docgen to refresh the full table after code changes.

See docs/research/multi_agent_surface_tasks/ for the full research record: experimental protocol, frozen results, the beam checkpoint-regression curve and payload-interface ablation, and the raw validation data.

Flight (`flight`)

Generic powered aerial flight profile with takeoff, transfer, and final landing

The Flight task is a generic lunar takeoff-and-landing aerial mission. The vehicle flies a staged three-phase profile over a 90 s fixed horizon: ascent to a cruise altitude, a planar transfer toward the landing pad, then a guarded approach into the lunar touchdown corridor.

Set env.landing_only=true to skip the ascent and transfer phases and start the mission directly in the terminal landing-descent phase (phase 2) — a descent-only scenario for isolating the touchdown problem. (This landing_only mode is a variant of the aerial Flight task, distinct from the separate orbital landing task.)


Episode Length	90.0s (2250 steps)
Reward Terms	7
Default Robot	Ingenuity

Compatible Robots

Crazyflie | Ingenuity

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.1	–
reward_waypoint_approach	Hyperbolic	12	4
reward_altitude_tracking	HyperbolicTracking	6	1
reward_waypoint_reached	Threshold	20	–
penalty_tilt_angle	OffsetLinear	-2	–
reward_terrain_clearance	HyperbolicBarrier	6	0.8
penalty_energy	Linear	-0.6	–

Formation Following (`formation_following`)

Multi-robot formation tracking behind a moving formation center

formation_following

Formation following is one of two centralized-policy multi-robot surface tasks in SRB (the companion task is beam transport). Three Cadre-class rovers track a slowly drifting formation center, each holding a fixed slot of a triangular pattern — offsets (1.5, 0), (-0.75, 1.3), (-0.75, -1.3) m, one per rover — while a single shared policy observes and drives the whole team. The center moves as a smooth random walk (or, with a ROS node attached, follows an external formation_center TF frame).

Centralized multi-agent formulation. One policy controls all three rovers: their per-robot observations are concatenated in a fixed, index-stable order into one joint observation, and the policy emits a single joint action that is split back across the rovers. Each environment returns one scalar reward for the team. This deliberately sidesteps decentralized communication and role assignment and keeps checkpoints reproducible, at the cost of joint observation and action dimensions that grow linearly with team size — a small-team baseline rather than a scalable coordination method.

# Train (DreamerV3 is the algorithm of record)
srb agent train --env formation_following

# Evaluate a trained checkpoint (reports the metrics below)
srb agent eval --algo dreamer --env formation_following --model <path/to/checkpoint>

The reference runs used 128 parallel environments for 30M agent steps at a 25 Hz control rate; episodes are 60 s (1500 steps). Train over independent random seeds (the committed configs do not pin one) and report mean ± standard deviation.

Evaluation. Each rollout reports the mean per-rover slot-distance error (formation_distance_error, m), the mean heading error (formation_orientation_error, rad), and the success-hold streak, alongside success_rate. Because no policy is trained yet, expect success_rate = 0; the distance and heading errors are the primary partial-competence signals and should be reported even at zero success.


Episode Length	60.0s (1500 steps)
Reward Terms	9
Default Robot	Cadre

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.1	–
penalty_formation_tracking	Linear	-1	–
reward_formation_progress	Scaled	4	–
reward_formation_velocity	Linear	2	–
reward_point_towards_target	Linear	0.5	–
reward_formation_precision	Linear	4	–
reward_in_zone	Linear	0	–
reward_formation_orientation	Linear	4	–
reward_action_rate_in_formation	Linear	32	–

Success criterion. An episode counts as a success once all three rovers simultaneously sit within formation_success_position_tolerance_xy = 0.20 m of their slots and within formation_success_yaw_tolerance ≈ 0.349 rad (20°) in heading, held for success_hold_steps = 8 consecutive control steps (0.32 s). The task is infinite-horizon (is_finite_horizon = false) and otherwise runs to the 60 s truncation.

Observation & action spaces. Per rover the policy observes the planar offset to its assigned slot and the heading-to-slot as a (sin, cos) pair, plus the shared formation-center planar velocity and each rover’s planar velocity. Every reset adds small fixed episodic sensing noise (1 cm position, 2.5° heading), so the policy cannot assume a noise-free slot estimate. The action is a planar velocity command (linear + angular) per rover.

Status and failure signature. Validation rollouts of the centralized-policy baselines reach zero task success, showing a wide, persistent gap: the rovers never approach the position or heading gate and tend to drift further over the rollout — consistent with a coordination behavior never discovered within the training budget (an exploration and curriculum-coverage gap). Unlike beam transport, this wide-gap signature does not by itself rule out a capacity or architecture limit, so capacity is not excluded here. The task is therefore a validated benchmark and workflow contribution: the interface, reward signals, logging, and reproducibility path are exercised end to end, while solving the task is open and gated on better reward shaping and training coverage.

Sim-to-real path. The task carries a deployment-facing interface: with a ROS node present the formation center is read from a formation_center TF frame instead of the internal random walk, and a unit-tested per-robot velocity and formation-center observation interface reconstructs the hardware-facing command and observation. Domain randomization (terrain, friction, sensing and actuation delays, command drift) is available as a staged transfer plan. No hardware or ROS deployment result is claimed.

A full research record for these two tasks — formulation, experimental protocol, the frozen zero-success results and per-task failure-mode analysis, the analysis figures, and the raw validation data — is kept in the repository under docs/research/multi_agent_surface_tasks/.

Formation Following Decentralized (`formation_following_decentralized`)

Single-rover policy view for decentralized formation following


Episode Length	60.0s (1500 steps)
Reward Terms	9
Default Robot	Cadre

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.1	–
penalty_formation_tracking	Linear	-1	–
reward_formation_progress	Scaled	4	–
reward_formation_velocity	Linear	2	–
reward_point_towards_target	Linear	0.5	–
reward_formation_precision	Linear	4	–
reward_in_zone	Linear	0	–
reward_formation_orientation	Linear	4	–
reward_action_rate_in_formation	Linear	32	–

Formation Following Orbital (`formation_following_orbital`)

Orbital formation tracking in microgravity

formation_following_orbital


Episode Length	60.0s (600 steps)
Reward Terms	7
Default Robot	Cubesat

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-8	–
penalty_fuel_consumption	Linear	0	–
penalty_action_magnitude	Linear	-8	–
penalty_formation_tracking	Linear	-2	–
reward_formation_precision	Linear	16	–
reward_formation_orientation	Linear	48	–
reward_action_rate_in_formation	Linear	48	–

Landing (`landing`)

Landing of spacecraft on planetary surfaces with varying terrain and obstacles


Episode Length	40.0s (1000 steps)
Reward Terms	11
Default Robot	Peregrine Lander

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.1	–
penalty_fuel_consumption	Linear	-1	–
penalty_angular_velocity	Linear	-0.25	–
penalty_gravity_rotation_alignment	Linear	-1	–
penalty_horizontal_linear_velocity	Linear	-0.1	–
reward_vertical_linear_velocity	Hyperbolic	8	1
reward_landing_angle	Linear	8	–
reward_landing_velocity	Linear	16	–
reward_landing_fuel	Linear	4	–
penalty_crash	Linear	-32	–
penalty_below_target	Linear	-16	–

Locomotion Velocity Tracking (`locomotion_velocity_tracking`)

Locomotion with legged robots and humanoids while tracking desired velocity commands


Episode Length	20.0s (500 steps)
Reward Terms	12
Default Robot	Spot

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.1	–
reward_cmd_lin_vel_xy	Linear	4	–
reward_cmd_ang_vel_z	Linear	2	–
penalty_undesired_lin_vel_z	Linear	-0.5	–
penalty_undesired_ang_vel_xy	Linear	-0.1	–
penalty_joint_torque	Linear	-0.000025	–
penalty_joint_acceleration	Linear	-0	–
penalty_undesired_robot_contacts	Threshold	-2	–
penalty_gravity_rotation_alignment	Linear	-2	–
reward_feet_air_time	Linear	0.75	–
reward_quadruped_trot	Linear	1	–
penalty_foot_slip	Linear	-0.5	–

Navigation to a dynamic waypoint with legged robots and humanoids


Episode Length	60.0s (1500 steps)
Reward Terms	13
Default Robot	Spot

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.1	–
penalty_position_tracking	Linear	-1	–
reward_point_towards_target	Linear	1	–
reward_position_tracking_precision	Linear	4	–
reward_orientation_tracking	Linear	8	–
reward_action_rate_at_target	Linear	16	–
penalty_joint_torque	Linear	-0.000025	–
penalty_joint_acceleration	Linear	-0	–
penalty_undesired_robot_contacts	Threshold	-2	–
penalty_undesired_lin_vel_z	Linear	-0.5	–
penalty_undesired_ang_vel_xy	Linear	-0.1	–
penalty_gravity_rotation_alignment	Linear	-2	–
reward_feet_air_time	Linear	0.5	–

Episodes terminate when a non-foot body contact exceeds the configured undesired-contact threshold or when projected gravity’s planar norm exceeds 0.85, indicating a fall. Non-finite contact or gravity state terminates fail-closed. Episode time limits remain truncations, not terminations.

Moonfall (`moonfall`)

Heavy-weight lunar descent with deployment stage and multi-stage landing phases


Episode Length	70.0s (1750 steps)
Reward Terms	12
Default Robot	Peregrine Lander

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.1	–
penalty_fuel_consumption	Linear	-1	–
penalty_angular_velocity	Linear	-0.25	–
penalty_gravity_rotation_alignment	Linear	-1	–
penalty_horizontal_linear_velocity	Linear	-0.1	–
reward_waypoint_progress	Hyperbolic	4	4
reward_vertical_linear_velocity	Hyperbolic	8	1
reward_landing_angle	Linear	8	–
reward_landing_velocity	Linear	16	–
reward_landing_fuel	Linear	4	–
penalty_crash	Linear	-32	–
penalty_below_target	Linear	-16	–

Orbital Evasion (`orbital_evasion`)

Navigation of spacecraft in microgravity while avoiding obstacles


Episode Length	30.0s (750 steps)
Reward Terms	7
Default Robot	Cubesat

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.1	–
penalty_fuel_consumption	Linear	-1	–
penalty_angular_velocity	Linear	-0.25	–
reward_obstacle_clearance	HyperbolicBarrier	4	2
penalty_distance_robot_to_target	Linear	-16	–
penalty_out_of_bounds	Linear	-16	–
reward_target_approach	Hyperbolic	8	5

Termination semantics. max_target_distance drives the penalty_out_of_bounds reward and does not set terminated. The physical collision threshold termination_min_obstacle_distance remains a genuine termination condition; the time limit is reported as truncated.

Navigation to a dynamic waypoint in orbital microgravity

orbital_waypoint_navigation


Episode Length	30.0s (300 steps)
Reward Terms	9
Default Robot	Satellite Mockup

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	LogCosh	-4	–
penalty_fuel_consumption	Linear	0	–
penalty_action_magnitude	LogCosh	-1	–
penalty_angular_velocity	LogCosh	-2	–
penalty_position_distance	Linear	-1	–
penalty_velocity_at_target	LogCosh	-4	–
reward_position_approach	Hyperbolic	4	2
reward_position_precision	Scaled	8	–
reward_smooth_station_keeping	Scaled	8	–

Rendezvous (`rendezvous`)

Maneuvering of spacecraft in microgravity to rendezvous with an uncooperative target


Episode Length	25.0s (625 steps)
Reward Terms	7
Default Robot	Cubesat

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.1	–
penalty_fuel_consumption	Linear	-8	–
penalty_angular_velocity	Linear	-0.05	–
penalty_distance_robot_to_target	Linear	-16	–
penalty_too_far	Linear	-16	–
reward_precision_position	Gaussian	128	0.025
reward_precision_orientation	Linear	128	–

Termination semantics. max_target_distance drives the penalty_too_far reward. Exceeding it leaves terminated false so the agent can recover; only the time limit sets truncated.

Terrain Landscaping (`terrain_landscaping`)

Terrain landscaping with a ground mobile robot reshaping a particle bed

terrain_landscaping

The default simulation uses a RaphRover with a separately articulated RaphShovel. The policy controls filtered linear/angular rover motion plus continuous shovel extension velocity through a three-dimensional action space. The same composition is available explicitly as env.robot=raph_rover+raph_shovel.

RaphShovel resets at its raised 0.2 m joint limit. This keeps the blade clear of the settled particle layer until the policy commands it downward; zero shovel velocity then holds the current position through actuator damping.

Launch direct teleoperation with one environment:

srb agent teleop -e terrain_landscaping env.num_envs=1

Keyboard controls use W/S for forward/reverse, C/V for rover yaw, and Q/E to raise/stow and lower/deploy the shovel. Releasing Q/E sends zero shovel velocity; the damped actuator resists drift without commanding an automatic retract. SpaceMouse forward/back and vertical translation control the same rover/shovel channels. Lateral translation is ignored by the non-holonomic rover.

Learning integrations receive the same ordered action vector: [rover_linear_velocity, rover_angular_velocity, shovel_linear_velocity].

Each episode draws its own mission. The target is a hashed, mass-balanced deformation of the flat reset bed: cut and fill volumes cancel to within 1e-4 m³, every requested cut stays above the available-material floor, every fill stays below the containment ceiling, and every significant deformation stays inside the reachable work zone. The mission is generated from a deterministic (env.general_mission_seed, env_id, episode_index) stream, so a recorded triple regenerates the identical mission in any process, and its manifest carries its own generator version, seed, frame, geometry, physical bounds, target-map hash, reset-layout hash, and manifest hash. A mission that cannot satisfy the declared envelope is rejected and redrawn; exhausting the attempt budget raises rather than emitting an infeasible target. The mission profile is Hydra-reachable under env.general_mission_profile.*.

This task deliberately shares no target, no manifest, and no curriculum stage with terrain_landscaping_crater: the two tasks share one reward implementation and one observation contract, and nothing else.


Episode Length	60.0s (600 steps)
Reward Terms	0
Default Robot	raph_rover

Note: The actor observation is produced exclusively through the external heightmap seam, not from a depth sensor. Every policy step the seam publishes one accepted-or-held 10 Hz frame of the work area, rasterized from the live regolith-particle positions (PhysX particle sets are not raycastable), and projects the seven contract leaves: current/target global maps (16×16), current/target local maps (24×16, rover body frame), the mission-frame pose, the body velocity, and the frame age in seconds. Map elevations are mission-frame +z; deploy adapters use the same vertical datum contract. The measurement model is selectable with env.external_heightmap.profile (ideal, randomized, failure_stale); only ideal has been exercised on this task.

The reward is the single stationary state-quality term 1 / (1 + mae / 0.02) evaluated on the mission’s fixed grading cells, the same implementation the crater variant uses. The MDP is continuing (is_finite_horizon=False); episodes end only by time-limit truncation and success is an evaluation metric reported there, never a termination or bonus.

The backward-compatible blade_pitch_normalized observation carries the active shovel joint state for the legacy deploy adapters; it is not an actor leaf. For RaphShovel it is normalized prismatic extension (0.0–0.2 m, raised/stowed at 0.2 m), not angular pitch. Legacy Leo rigid/tilt payload configurations remain available for existing policies and real-hardware adapters.

The seam’s timing contract is fixed: one accepted-or-held frame per policy step at 10 Hz, with the map, mission pose and body velocity captured in one snapshot under a single source timestamp (the local crop is sampled at that frame’s own pose, never an older map with a newer pose). Reset blocks up to 2.0 s for the first complete frame and raises rather than substituting a zero frame; a frame older than the 0.5 s hard cutoff is a stale abort outside the MDP; a dropped update re-publishes the previous complete map and raises the age value rather than creating holes. There is no cell-validity mask, so full finite coverage of the 146×146 source is a hard invariant.

Because shovel extension is unobserved, TD-MPC2 is disabled for this task and fails closed before any expensive side effect. No Dreamer learnability evidence exists for it either.

This is simulation-only training geometry, no real-hardware validation exists, and no learned policy is claimed for it: the scripted-reference environment-reachability gate for this task does not pass — the reference finishes 0.18–0.37 mm worse than zero-action on all three declared missions, against a required 2.0 mm improvement (see docs/superpowers/evidence/terrain_landscaping/A10.md and Terrain Landscaping Training Readiness).

Terrain Landscaping Crater (`terrain_landscaping_crater`)

Separate deterministic baseline: fill one shallow crater from a nearby regolith pile

This is a separate task, not a curriculum stage of terrain_landscaping. There is no automatic difficulty progression, no scheduler, no implicit switching between the two task IDs, and no claim that a crater-trained model solves general landscaping. Each task carries its own readiness claim, and a result on one is not evidence for the other in either direction.

This deterministic baseline keeps only geometry needed for the first training problem: a layered particle bed with one shallow bowl crater and one loose regolith pile. Particles removed from the crater are reused in the pile, so initial particle count and mass stay constant. The RaphRover starts at a fixed pose with both features inside its forward work area.

The work area uses a visible 4 m slab below the physical ground plane. Dark neutral terrain and lighter regolith keep the crater floor, pile, and remaining bed readable against the lunar skydome in both the viewport and headless captures.

The target is the static analytic flat-bed map materialized from the checked-in target manifest (crater_target_manifest.json); configuration fails if any crater/pile/spawner field stops reproducing the manifest’s hashed reset layout. The reward is the single stationary state-quality term 1 / (1 + mae / 0.02) evaluated on the manifest’s fixed grading cells from oracle particle truth; success (90 % of grading cells within tolerance for 20 consecutive newly accepted mapper frames) is an evaluation metric reported at the time-limit truncation, never a termination or bonus. The MDP is continuing (is_finite_horizon=False); episodes end only by truncation. Physics perturbations and lighting randomization are disabled in this first baseline. The action and observation schemas remain compatible with terrain_landscaping.

Inspect it with direct teleoperation:

srb agent teleop -e terrain_landscaping_crater env.num_envs=1

Before starting any training run here, read Terrain Landscaping Training Readiness. No learned policy is claimed for this task. The scripted-reference environment-reachability gate does not pass — ten distinct controller architectures reached at best +0.31 mm against a required 2.0 mm margin (the ten ran in separate processes, whose ~0.23 mm settled-state spread makes their ranking unresolvable — the ~1.5 mm shortfall is not) — and the bounded Dreamer learnability gate could not be measured at all: 19 of 21 seeded 10,000-step canary attempts died on a mission-pose envelope abort, and the one surviving checkpoint finished worse than doing nothing on every evaluation seed. That abort is learner-independent — it is raised by the environment’s own frame validation (MeasuredExternalHeightmapSource._validate_frame), and it has already killed a scripted-controller run and 1 of 2 attempts of a 1,000-step Dreamer smoke — so the run below is exposed to exactly the same failure, and will die the same way whenever the rover’s centre crosses the bed edge. The command below runs; it is not evidence that anything learns.

Start a baseline training run:

srb agent train --headless --algo skrl_ppo -e terrain_landscaping_crater


Episode Length	30.0s (300 steps)
Reward Terms	0
Default Robot	raph_rover

Crater center/radius/depth default to (0.45 m, 0.20 m), 0.36 m, and 0.11 m. Pile center/radius/height default to (-0.25 m, -0.20 m), 0.32 m, and 0.28 m. These fields are Hydra-overridable under env.*; invalid out-of-bed, overlapping, or under-capacity layouts fail before particle spawn.

Note: The actor observation is the same external-heightmap contract the general task uses, and it comes from no depth sensor. Every policy step the seam publishes one accepted-or-held 10 Hz frame rasterized from the live regolith-particle positions and projects exactly seven leaves: current and target global maps (16×16), current and target local maps (24×16, rover body frame), the mission-frame pose (x, y, yaw), the body velocity (vx, vy, wz), and the frame age in seconds — flattened to exactly 1287 float32 in that order. There is no cell-validity mask, so full finite coverage of the 146×146 source is a hard invariant and an incomplete or out-of-envelope frame is rejected whole rather than filled. Shovel extension, previous action, particle positions, the simulator world pose, and privileged current-minus-target maps are deliberately not observed. The measurement model is selected with env.external_heightmap.profile (ideal, randomized, failure_stale); only ideal has been exercised on a training run. A frame older than the 0.5 s hard cutoff is rejected as an out-of-MDP stale abort, and reset blocks up to 2.0 s for the first complete frame.

Because shovel extension is unobserved and no approved history/recurrent state contract exists, TD-MPC2 is disabled for this task and fails closed with an actionable error before any log directory, config, replay or model is created. Dreamer’s integration is proven for one environment (env.num_envs=1; num_envs > 1 is refused fail-closed), but its learnability gate is blocked — see Terrain Landscaping Training Readiness.

This is simulation-only training geometry. It does not prove regolith fidelity or physical-rover readiness, and no real-hardware validation exists for it. A CUDA-capable NVIDIA GPU remains required for PhysX particle dynamics: the PBD solver is GPU-only, while the per-particle readback SRB consumes is the CPU-facing USD transport it feeds. The task deliberately uses the CPU simulation readback pipeline because Isaac Sim 6’s CUDA direct-data pipeline does not synchronize PBD particle positions or velocities back to the USD points consumed by the heightmap, reward, and renderer.

Terrain Landscaping Mound (`terrain_landscaping_mound`)

Separate deterministic baseline: level one regolith mound on a flat bed, graded by excess volume


Episode Length	30.0s (300 steps)
Reward Terms	0
Default Robot	raph_rover

Traversal (`traversal`)

Straight-line path following at a target velocity

traversal


Episode Length	120.0s (3000 steps)
Reward Terms	5
Default Robot	Nifty

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.05	–
penalty_lateral_pos_deviation	Linear	-1	–
reward_vel_progress	Linear	8	–
reward_action_rate_at_target	Linear	32	–
penalty_rollover	Linear	-16	–

Termination semantics. max_tilt_rad drives the penalty_rollover reward. Exceeding the tilt threshold leaves terminated false so the rover can recover; the time limit is reported as truncated.

Velocity Tracking (`velocity_tracking`)

Basic tracking of desired velocity commands


Episode Length	20.0s (500 steps)
Reward Terms	5
Default Robot	Cadre

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.1	–
reward_cmd_lin_vel_xy	Linear	4	–
reward_cmd_ang_vel_z	Linear	2	–
penalty_undesired_lin_vel_z	Linear	-0.5	–
penalty_undesired_ang_vel_xy	Linear	-0.1	–

Navigation to a dynamic waypoint


Episode Length	60.0s (1500 steps)
Reward Terms	6
Default Robot	Cadre

Compatible Robots

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.1	–
penalty_position_tracking	Linear	-1	–
reward_point_towards_target	Linear	1	–
reward_position_tracking_precision	Linear	4	–
reward_orientation_tracking	Linear	8	–
reward_action_rate_at_target	Linear	16	–

Environments – Mobile Manipulation

Environments combining mobility and manipulation capabilities.

Aerial Manipulation (`_aerial_manipulation`)

Template environment for aerial mobile manipulators. Configure via env.robot.mobile_base=<aerial_robot> and env.robot.manipulator=<manipulator>.

_aerial_manipulation


Episode Length	10.0s (500 steps)
Reward Terms	2
Default Robot	Generic Aerial Manipulator

Compatible Robots

Generic Aerial Manipulator

Reward Breakdown

Term	Type	Weight	Scale	Notes
penalty_action_rate	Linear	-0.1	–
penalty_undesired_robot_contacts	Threshold	-0.5	–

Ground Manipulation (`_ground_manipulation`)

Template environment for ground mobile manipulators. Configure via env.robot.mobile_base=<ground_robot> and env.robot.manipulator=<manipulator>.

_ground_manipulation


Episode Length	10.0s (500 steps)
Reward Terms	2
Default Robot	Generic Ground Manipulator

Compatible Robots

Generic Ground Manipulator

Reward Breakdown

Term	Type	Weight	Scale	Notes
penalty_action_rate	Linear	-0.1	–
penalty_undesired_robot_contacts	Threshold	-0.5	–

Orbital Manipulation (`_orbital_manipulation`)

Template environment for orbital mobile manipulators. Configure via env.robot.mobile_base=<orbital_robot> and env.robot.manipulator=<manipulator>.

_orbital_manipulation


Episode Length	10.0s (500 steps)
Reward Terms	2
Default Robot	Generic Orbital Manipulator

Compatible Robots

Generic Orbital Manipulator

Reward Breakdown

Term	Type	Weight	Scale	Notes
penalty_action_rate	Linear	-0.1	–
penalty_undesired_robot_contacts	Threshold	-0.5	–

Mobile Debris Capture (`mobile_debris_capture`)

Capture of floating debris in microgravity with a floating-base mobile manipulator


Episode Length	60.0s (3000 steps)
Reward Terms	10
Default Robot	Generic Orbital Manipulator

Compatible Robots

Generic Orbital Manipulator

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.1	–
penalty_fuel_consumption	Linear	-2	–
penalty_angular_velocity	Linear	-0.05	–
penalty_joint_torque	Linear	-0.000025	–
penalty_joint_acceleration	Linear	-0.0005	–
reward_distance_end_effector_to_obj	Gaussian	16	0.2
reward_grasp	SoftThreshold	32	–
reward_minimize_rel_lin_vel	Gaussian	12	0.01
reward_minimize_rel_ang_vel	Gaussian	24	0.03
penalty_debris_too_far	Linear	-16	–

Termination semantics. distance_debris_too_far drives the penalty_debris_too_far reward and does not set terminated; the finite horizon is reported as truncated.

Spacewalk (`spacewalk`)

Humanoid spacewalk: rail-aligned waypoint traversal near spacecraft structure

spacewalk

Spacewalk casts microgravity humanoid traversal as a contact-rich locomotion problem: a thruster-less humanoid reaches a sequence of waypoints near a spacecraft and station-keeps at the final one, propelling itself by pushing off the structure and handrails as an EVA astronaut does (see Design notes for the physics). It ships in two modes — exterior EVA and interior IVA.

Use env.mode=eva for exterior ISS+Gateway, handrail-adjacent traversal (default), or env.mode=iva for inside-Kibou training. The base spacewalk env honours env.mode: setting env.mode=iva automatically retargets the waypoint path to the Kibou interior (IVA_WAYPOINT_PATH). The dedicated spacewalk_eva / spacewalk_iva aliases pre-select each mode and are the recommended way to keep the split explicit in benchmarks and launch configs. The two modes also differ substantially in traversal character — a multi-metre exterior leg with a lateral turn versus a short confined-space acquisition — as detailed under Locomotion below.

EVA and IVA share a single waypoint source of truth (EVA_WAYPOINT_PATH / IVA_WAYPOINT_PATH), from which the scenery handrails are also derived. (This automatic link applies to the built-in paths; a custom waypoint_path should be paired with a matching env.scenery.handrail_segment_specs, since the rails do not otherwise follow an overridden path.) The distance-to-nearest-handrail observation is always present, so the observation space is identical across modes — but it is a meaningful rail distance only in EVA. The IVA Kibou interior has no handrails, so there the value is the constant no_handrail_distance fallback — in IVA the agent cannot sense its propulsion surfaces (the module walls) and must push off reactively from IMU-sensed contact, which the confined interior keeps feasible.

Setting env.use_waypoint_path=false switches from path traversal to a single randomly placed target (within ±target_pos_range m of the spacecraft origin) — a simpler reaching variant useful for ablations.

The RL algorithm of record is DreamerV3 (hyperparams/dreamerv3.yaml); validation specs live at hyperparams/validation/spacewalk*.yaml. A policy must be trained first — no trained checkpoint ships yet:

# Train (DreamerV3 is the algorithm of record)
srb agent train --env spacewalk_eva

# Evaluate a trained checkpoint (reports the metrics below)
srb agent eval --algo dreamer --env spacewalk_eva --model <path/to/checkpoint>

Benchmark protocol (intended; no results ship yet — this is a draft). The protocol follows the committed configs so it is reproducible once policies are trained.

Training. DreamerV3 with num_envs = 16 parallel environments, world model deter = 512, stochastic latent 32 × 32, imagination horizon = 15, replay capacity 1e6, replay ratio = 512, up to max_iterations = 2e6 policy steps (all in hyperparams/dreamerv3.yaml). These are the shared SRB orbital-task defaults (the spacewalk block is identical to the excavation/terrain_landscaping blocks), not tuned for spacewalk’s harder contact-locomotion regime — expect to tune them when training in earnest. Repeat training over independent random seeds — supplied per run, since the committed configs do not pin a training seed — and report mean ± standard deviation.

Evaluation. n_episodes = 10 rollouts per checkpoint (hyperparams/validation/spacewalk*.yaml), reporting five metrics:

success_rate — fraction of episodes that reached and held the final waypoint.
min_distance_to_target — closest approach achieved to the current target waypoint over the episode, metres (small once any waypoint is neared).
min_distance_to_final_target — closest approach to the final waypoint over the episode (a clean completion measure).
waypoint_progress — normalised advancement along the path, index / (num_waypoints − 1).
dist_to_nearest_handrail — body-to-rail distance (meaningful in EVA).

(The real-hardware validation spec’s metrics_to_track is a superset of this reported set: it additionally records the generic validation-harness metrics terminal_reward, action_smoothness, and wall_seconds.)

Interpreting near-zero success. Because contact-based locomotion is hard to discover, early policies may score near-zero success_rate; min_distance_to_target and waypoint_progress are the primary signals of partial competence and should be reported even when success is zero, rather than collapsing the result to a single binary rate. Read together they localise the bottleneck:

low waypoint_progress with a large min_distance_to_target — the agent never gets close to even the first waypoint (push-off discovery fails);
low waypoint_progress with a small min_distance_to_target — it nears a waypoint but cannot reliably advance;
waypoint_progress near 1.0 with zero success_rate — the failure is on the final leg. min_distance_to_final_target then separates the two sub-cases: still above success_radius means the agent never reached the final waypoint, whereas near zero with zero success_rate means it reached the final waypoint but could not hold the station-keep (the terminal velocity-arrest).

For EVA, ISSGatewayScenery can be customized without editing code:

env.scenery.station_template_usd_path — replace ISS+Gateway pair with a single USD station file.
env.scenery.use_exact_station_mesh_collision — enable exact triangular mesh collision (default: false).
env.scenery.add_handrails / env.scenery.handrail_segment_specs — control optional support rails aligned to the waypoint path.
env.scenery.handrail_specs — legacy shorthand for X-axis rails.

An episode is reported as a success once the humanoid reaches the final waypoint (within success_radius = 0.5 m of it) and remains inside that radius for success_stability_steps = 10 consecutive control steps. The success flag is sticky: once set, it stays set for the rest of the episode, so the metric counts “reached and briefly held the goal”, not “ended the episode at the goal”.

Each episode resets the humanoid within a ±0.5 m box (at the EVA path start, or about the Kibou origin for IVA) with fully random orientation and a small random velocity (±0.1 m/s linear, ±5°/s angular) — so a policy cannot assume a fixed start pose or a zero initial drift. Physics and control run at the same 25 Hz (decimation 1, dt = 40 ms), so the push-off contact dynamics the locomotion relies on are resolved at the relatively coarse control step — relevant to both reproduction fidelity and the difficulty of learning stable contacts.

Observation & action spaces. The policy observes (DreamerV3 vector inputs): robot orientation as a 6D rotation (tf_rot6d_robot), the robot→target-waypoint vector in the robot’s body frame (tf_pos_robot_to_target, so the target is seen egocentrically), scalar distances to the spacecraft origin (dist_to_spacecraft) and to the nearest handrail (dist_to_nearest_handrail), normalised traversal progress (waypoint_progress), body-frame linear and angular velocity, joint positions and velocities, and IMU linear-acceleration / angular-velocity. The action is per-joint position targets for the selected humanoid; the action dimensionality equals the robot’s actuated-joint count, resolved at runtime from the USD articulation. Targets are bounded to each joint’s physical position limits (JointPositionToLimits), so the policy cannot command out-of-range configurations — unlike the raw-scale observations, the action space is intrinsically bounded. There is no explicit contact-state observation — the agent infers surface contact (needed to time push-offs, per the locomotion notes below) from the IMU’s linear acceleration; adding an explicit contact signal is a natural future-work lever.

Observations are fed at their raw scales (metre-scale distances alongside the [0,1] waypoint_progress and [-1,1] rot6d); the task applies no manual normalisation because DreamerV3’s built-in symlog encoding absorbs that dynamic range. A non-DreamerV3 algorithm should add its own observation normalisation.


Episode Length	20.0s (500 steps)
Reward Terms	9
Default Robot	Unitree G1

Compatible Robots

Humanoid21 | Humanoid28 | Unitree H1 | Unitree G1

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.5	–
penalty_angular_velocity	Linear	-2	–
reward_distance_to_target	Gaussian	16	2.5
reward_progress	Scaled	8	–
reward_approach_velocity	GaussianAlignment	12	0.7854
reward_reaching	SoftThreshold	32	–
reward_stability	Gaussian	8	0.1
reward_success_hold	Linear	10	–
penalty_drift_from_spacecraft	SoftThreshold	-6	–

These are configuration-compatible embodiments, not validated publication baselines. Unitree G1 remains the tested publication default for the current Spacewalk draft.

Reward design. The nine terms compose into four functional groups: (1) long-range guidance — reward_distance_to_target (Gaussian, scale 2.5), reward_approach_velocity (direction alignment), and reward_progress (signed per-step delta) shape the approach from far out; (2) terminal acquisition — reward_reaching (sharp soft-threshold at a fixed 0.5 m reaching radius, independent of the configurable success_radius — they coincide only at the default success_radius = 0.5) becomes the dominant signal at the goal; (3) station-keeping — reward_stability (low-velocity Gaussian, gated to the success region) and reward_success_hold (streak fraction) reward holding the final waypoint; (4) regularisation and safety — the action-rate and angular-velocity penalties damp jittery actuation, and the one-sided penalty_drift_from_spacecraft keeps the agent within the working envelope — the ≤~7 m-from-origin region the waypoints span, beyond which the penalty’s 9 m activation knee deliberately sits so it never fires during legitimate traversal. Drifting off-structure is effectively unrecoverable (a thruster-less humanoid with no surface to push from cannot propel back), so the penalty discourages escape without pulling the agent toward the origin. The net effect is a monotonic landscape — total reward rises from drifting away, through productive traversal, to holding at the goal. Unit tests pin this landscape and check the reward resists common reward-hacking patterns: in-place spinning (defeated by the angular penalty, since the stability term gates on linear velocity only), toward/away oscillation, and hovering just outside the success boundary.

One shaping limitation is known and left as future work. The dense reward_distance_to_target and reward_reaching terms track the current waypoint, which teleports to the next leg the instant an intermediate waypoint is entered (and there is no explicit per-advance bonus — waypoint_progress is only observed/reported, never rewarded). Crossing an intermediate waypoint therefore causes a transient per-step reward drop of roughly the distance+reaching value of the leg just closed. The coarse landscape above still favours reaching and holding the final goal (the global optimum), but a farsighted agent must see past this local dip rather than camp just outside each intermediate radius. A characterisation test (test_intermediate_waypoint_advance_has_known_reward_cliff) pins the current magnitude; the natural fix — folding waypoint_progress into the reward or using a potential-based path-following term — changes the reward landscape and so needs GPU re-validation before it ships.

Design notes. Episodes are fixed-horizon (20 s / 500 steps) with truncation only — there is no hard failure termination, so every rollout runs the full horizon. The horizon is a truncation, not a terminal (termination is always false in the step return): a correct agent bootstraps its value estimate at the time limit rather than treating reaching it as a failure — important to handle right when reproducing with a non-DreamerV3 algorithm. The reward shapes traversal alone: there is no contact or collision penalty, even though the station, Gateway, and handrails are rigid colliders. An agent is therefore neither rewarded for grasping a handrail nor penalised for brushing the structure — the benchmark scores rail-adjacent waypoint traversal and station-keeping, not contact-aware EVA. Natural future-work extensions: a collision penalty targeting high-impact contact (distinct from the controlled push-off the agent relies on to move — penalising all contact would break locomotion), an explicit handrail-grasp objective, and an explicit contact-state observation.

Locomotion. The agent commands only humanoid joint positions — there are no thrusters and no ground in the microgravity scene. Internal joint motion cannot change the body’s centre-of-mass velocity (momentum is conserved), so to translate toward distant waypoints the humanoid must push or pull against the station and handrails, as an EVA astronaut does; the small randomised initial drift (±0.1 m/s) covers only a fraction of a multi-metre leg. This gives the handrails a triple role — distance cue, propulsion surface, and (the EVA path terminates at a rail) a brace for arresting velocity to station-keep at the goal. It is also why structure contact is left unpenalised, and it makes the benchmark a hard contact-rich locomotion problem — part of why no policy is trained yet. The reward credits the outcome of locomotion (approach velocity, progress, distance), not the contact event itself, so push-off must be discovered from its downstream effect rather than followed up a local gradient; densifying that signal with a contact/grasp-shaping term (the future work noted above) is the natural way to make the task more learnable.

EVA and IVA also differ in character: EVA is a multi-metre exterior traversal along handrails with a lateral turn, whereas IVA is a short confined-space task inside Kibou. The ~12.5 m EVA path within the 20 s horizon implies a ~0.6 m/s required average, so push-off chaining must be efficient — there is little slack for dawdling before the episode truncates. Because the shared success_radius (0.5 m) is comparable to the interior waypoint spacing, IVA mainly exercises final-waypoint acquisition and station-keeping in a confined volume rather than long-range traversal.

Spacewalk Eva (`spacewalk_eva`)

Humanoid EVA traversal along exterior spacecraft handrails


Episode Length	20.0s (500 steps)
Reward Terms	9
Default Robot	Unitree G1

Compatible Robots

Humanoid21 | Humanoid28 | Unitree H1 | Unitree G1

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.5	–
penalty_angular_velocity	Linear	-2	–
reward_distance_to_target	Gaussian	16	2.5
reward_progress	Scaled	8	–
reward_approach_velocity	GaussianAlignment	12	0.7854
reward_reaching	SoftThreshold	32	–
reward_stability	Gaussian	8	0.1
reward_success_hold	Linear	10	–
penalty_drift_from_spacecraft	SoftThreshold	-6	–

Spacewalk Iva (`spacewalk_iva`)

Humanoid IVA traversal inside the Kibou module


Episode Length	20.0s (500 steps)
Reward Terms	9
Default Robot	Unitree G1

Compatible Robots

Humanoid21 | Humanoid28 | Unitree H1 | Unitree G1

Reward Breakdown

Term	Type	Weight	Scale
penalty_action_rate	Linear	-0.5	–
penalty_angular_velocity	Linear	-2	–
reward_distance_to_target	Gaussian	16	2.5
reward_progress	Scaled	8	–
reward_approach_velocity	GaussianAlignment	12	0.7854
reward_reaching	SoftThreshold	32	–
reward_stability	Gaussian	8	0.1
reward_success_hold	Linear	10	–
penalty_drift_from_spacecraft	SoftThreshold	-6	–

Environments – Digital Twins

High-fidelity digital twin environments modelling real-world facilities and testbeds.

Lunalab (`lunalab`)

Digital twin of the LunaLab planetary analogue testbed

lunalab lunalab


Episode Length	100000.0s (2500000 steps)
Reward Terms	2
Default Robot	Generic Ground Manipulator

Compatible Robots

Generic Ground Manipulator

Reward Breakdown

Term	Type	Weight	Scale	Notes
penalty_action_rate	Linear	-0.1	–
penalty_undesired_robot_contacts	Threshold	-0.5	–

Zerog Lab (`zerog_lab`)

Digital twin of a zero-gravity laboratory testbed

zerog_lab


Episode Length	100000.0s (1000000 steps)
Reward Terms	2
Default Robot	Generic Orbital Manipulator

Compatible Robots

Generic Orbital Manipulator

Reward Breakdown

Term	Type	Weight	Scale	Notes
penalty_action_rate	Linear	-0.1	–
penalty_fuel_consumption	Linear	-1	–

Assets

Overview

This section documents all robots, objects, and scenery available in the Space Robotics Bench.

Robot Arms & Manipulators
Mobile Robots
Mobile Manipulators
End Effectors & Tools
Objects
Scenery

Robot Arms & Manipulators

Serial-link robot arms registered in the Space Robotics Bench. These manipulators are used for dexterous manipulation, assembly, and maintenance tasks across terrestrial and space environments.

Overview

Franka

Franka (`franka`)

7-DoF torque-controlled research arm from Franka Robotics

franka franka


DoF	7
End Effector	FrankaHand
Base Frame	`panda_link0`
Flange Frame	`panda_link7`
Sensors	3
Action Modes	5 (default: `diff_ik`)
Source	SRB

Action Modes

Mode	Controller	Dims	Scale
`diff_ik` *	Differential Inverse Kinematics	6	0.1
`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
`joint_pos_rel`	Relative Joint Position	7	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
panda_shoulder	`panda_joint[1-4]`	87 N·m	2.175 rad/s	4000	800
panda_forearm	`panda_joint[5-7]`	12 N·m	2.61 rad/s	4000	800

Sensors

Name	Type	Location
contacts_robot	Contact Sensor	`.*`
cam_base	Camera	`panda_link0/camera_base`
cam_wrist	Camera	`panda_link7/camera_wrist`

Initial Joint Configuration

Joint	Position
panda_joint1	0.0 rad
panda_joint2	0.0 rad
panda_joint3	0.0 rad
panda_joint4	-1.570796 rad (-90 deg)
panda_joint5	0.0 rad
panda_joint6	1.570796 rad (90 deg)
panda_joint7	0.785398 rad (45 deg)

Kinova

Kinova Gen3N7 (`kinova_gen3n7`)

7-DoF lightweight research arm from Kinova Robotics

kinova_gen3n7


DoF	7
End Effector	None
Base Frame	`base_link`
Flange Frame	`end_effector_link`
Sensors	3
Action Modes	6 (default: `diff_ik`)
Source	Isaac Lab

Action Modes

Mode	Controller	Dims	Scale
`diff_ik` *	Differential Inverse Kinematics	6	0.1
`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
`smooth_osc`	Smooth Operational Space Controller	6	pos=0.1, ori=0.1
`joint_pos_rel`	Relative Joint Position	7	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
arm	`joint_[1-7]`	2340 N·m	10 rad/s	500	200

Sensors

Name	Type	Location
contacts_robot	Contact Sensor	`.*`
cam_base	Camera	`base_link/camera_base`
cam_wrist	Camera	`end_effector_link/camera_wrist`

Initial Joint Configuration

Joint	Position
joint_1	0.0 rad
joint_2	0.785398 rad (45 deg)
joint_3	0.0 rad
joint_4	0.785398 rad (45 deg)
joint_5	0.0 rad
joint_6	1.570796 rad (90 deg)
joint_7	0.0 rad

Kinova J2N6S (`kinova_j2n6s`)

kinova_j2n6s


DoF	6
End Effector	Kinova300
Base Frame	`j2n6s_link_base`
Flange Frame	`j2n6s_link_6`
Sensors	3
Action Modes	2 (default: `diff_ik`)

Action Modes

Mode	Controller	Dims	Scale
`diff_ik` *	Differential Inverse Kinematics	6	0.1
`joint_pos_rel`	Relative Joint Position	6	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
arm	`j2n6s_joint_[1-6]`	80 N·m	100 rad/s	4000	1000

Sensors

Name	Type	Location
contacts_robot	Contact Sensor	`.*`
cam_base	Camera	`j2n6s_link_base/camera_base`
cam_wrist	Camera	`j2n6s_link_6/camera_wrist`

Initial Joint Configuration

Joint	Position
j2n6s_joint_1	0.0 rad
j2n6s_joint_2	2.617994 rad (150 deg)
j2n6s_joint_3	5.759587 rad (330 deg)
j2n6s_joint_4	-1.570796 rad (-90 deg)
j2n6s_joint_5	0.0 rad
j2n6s_joint_6	0.0 rad

Kinova J2N7S (`kinova_j2n7s`)

kinova_j2n7s


DoF	7
End Effector	Kinova300
Base Frame	`j2n7s_link_base`
Flange Frame	`j2n7s_link_7`
Sensors	3
Action Modes	2 (default: `diff_ik`)

Action Modes

Mode	Controller	Dims	Scale
`diff_ik` *	Differential Inverse Kinematics	6	0.1
`joint_pos_rel`	Relative Joint Position	7	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
arm	`j2n7s_joint_[1-7]`	80 N·m	100 rad/s	4000	1000

Sensors

Name	Type	Location
contacts_robot	Contact Sensor	`.*`
cam_base	Camera	`j2n7s_link_base/camera_base`
cam_wrist	Camera	`j2n7s_link_7/camera_wrist`

Initial Joint Configuration

Joint	Position
j2n7s_joint_1	0.0 rad
j2n7s_joint_2	2.617994 rad (150 deg)
j2n7s_joint_3	0.0 rad
j2n7s_joint_4	0.523599 rad (30 deg)
j2n7s_joint_5	1.570796 rad (90 deg)
j2n7s_joint_6	0.0 rad
j2n7s_joint_7	0.0 rad

Multi-Arm

Levion Arms (`levion_arms`)

levion_arms


DoF	4
End Effector	None
Base Frame	`base_link`
Flange Frame	–
Sensors	0
Action Modes	default: `dual_arm_joint_pos_abs`

Action Modes

Mode	Controller	Dims	Scale
`dual_arm_joint_pos_abs` *	Joint Position To Limits	4	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
left_arm	`left_shoulder_joint`, `left_elbow_joint`	15 N·m	–	400	80
right_arm	`right_shoulder_joint`, `right_elbow_joint`	15 N·m	–	400	80

Initial Joint Configuration

Joint	Position
right_elbow_joint	0.4 rad (22.918312 deg)
left_elbow_joint	-0.4 rad (-22.918312 deg)

Zeroglab Arms (`zeroglab_arms`)

zeroglab_arms


DoF	7
End Effector	None
Base Frame	`_virtual_world`
Flange Frame	–
Sensors	0
Action Modes	default: `dual_arm_osc`

Action Modes

Mode	Controller	Dims	Scale
`dual_arm_osc` *	Operational Space Controller	24	arm_ceiling: pos=0.1, ori=0.1, stiffness=120.0, arm_wall: pos=0.1, ori=0.1, stiffness=120.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
rail	`rail_joint`	400 N·m	0.2 rad/s	3000	2500
arm	`.shoulder.`, `.elbow.`, `.wrist.`	87 N·m	100 rad/s	800	40

Initial Joint Configuration

Joint	Position
rail_joint	2 rad (114.591559 deg)
shoulder_pan_joint	0.0 rad
shoulder_lift_joint	-1.570796 rad (-90 deg)
elbow_joint	1.570796 rad (90 deg)
wrist_1_joint	-1.570796 rad (-90 deg)
wrist_2_joint	-1.570796 rad (-90 deg)
wrist_3_joint	-1.570796 rad (-90 deg)

Other

Random Ur Manipulator (`random_ur_manipulator`)

random_ur_manipulator


DoF	6
End Effector	None
Base Frame	`base_link`
Flange Frame	`wrist_3_link`
Sensors	3
Action Modes	5 (default: `diff_ik`)

Action Modes

Mode	Controller	Dims	Scale
`diff_ik` *	Differential Inverse Kinematics	6	0.1
`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
`joint_pos_rel`	Relative Joint Position	6	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
arm	`.*`	87 N·m	100 rad/s	800	40

Sensors

Name	Type	Location
contacts_robot	Contact Sensor	`.*`
cam_base	Camera	`base_link/camera_base`
cam_wrist	Camera	`wrist_3_link/camera_wrist`

Initial Joint Configuration

Joint	Position
shoulder_pan_joint	0.0 rad
shoulder_lift_joint	-1.570796 rad (-90 deg)
elbow_joint	1.570796 rad (90 deg)
wrist_1_joint	-1.570796 rad (-90 deg)
wrist_2_joint	-1.570796 rad (-90 deg)
wrist_3_joint	-1.570796 rad (-90 deg)

Ur10 (`ur10`)

Heavy-duty collaborative arm from Universal Robots

ur10 ur10


DoF	6
End Effector	None
Base Frame	`base_link`
Flange Frame	`ee_link`
Sensors	3
Action Modes	5 (default: `diff_ik`)
Source	Isaac Lab

Action Modes

Mode	Controller	Dims	Scale
`diff_ik` *	Differential Inverse Kinematics	6	0.1
`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
`joint_pos_rel`	Relative Joint Position	6	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
arm	`.*`	87 N·m	100 rad/s	800	40

Sensors

Name	Type	Location
contacts_robot	Contact Sensor	`.*`
cam_base	Camera	`base_link/camera_base`
cam_wrist	Camera	`ee_link/camera_wrist`

Initial Joint Configuration

Joint	Position
shoulder_pan_joint	0.0 rad
shoulder_lift_joint	-1.570796 rad (-90 deg)
elbow_joint	1.570796 rad (90 deg)
wrist_1_joint	-1.570796 rad (-90 deg)
wrist_2_joint	-1.570796 rad (-90 deg)
wrist_3_joint	-1.570796 rad (-90 deg)

Ur10E (`ur10e`)

ur10e ur10e


DoF	6
End Effector	None
Base Frame	`base_link`
Flange Frame	`wrist_3_link`
Sensors	3
Action Modes	5 (default: `diff_ik`)

Action Modes

Mode	Controller	Dims	Scale
`diff_ik` *	Differential Inverse Kinematics	6	0.1
`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
`joint_pos_rel`	Relative Joint Position	6	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
arm	`.*`	87 N·m	100 rad/s	800	40

Sensors

Name	Type	Location
contacts_robot	Contact Sensor	`.*`
cam_base	Camera	`base_link/camera_base`
cam_wrist	Camera	`wrist_3_link/camera_wrist`

Initial Joint Configuration

Joint	Position
shoulder_pan_joint	0.0 rad
shoulder_lift_joint	-1.570796 rad (-90 deg)
elbow_joint	1.570796 rad (90 deg)
wrist_1_joint	-1.570796 rad (-90 deg)
wrist_2_joint	-1.570796 rad (-90 deg)
wrist_3_joint	-1.570796 rad (-90 deg)

Ur16E (`ur16e`)

ur16e ur16e


DoF	6
End Effector	None
Base Frame	`base_link`
Flange Frame	`wrist_3_link`
Sensors	3
Action Modes	5 (default: `diff_ik`)

Action Modes

Mode	Controller	Dims	Scale
`diff_ik` *	Differential Inverse Kinematics	6	0.1
`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
`joint_pos_rel`	Relative Joint Position	6	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
arm	`.*`	87 N·m	100 rad/s	800	40

Sensors

Name	Type	Location
contacts_robot	Contact Sensor	`.*`
cam_base	Camera	`base_link/camera_base`
cam_wrist	Camera	`wrist_3_link/camera_wrist`

Initial Joint Configuration

Joint	Position
shoulder_pan_joint	0.0 rad
shoulder_lift_joint	-1.570796 rad (-90 deg)
elbow_joint	1.570796 rad (90 deg)
wrist_1_joint	-1.570796 rad (-90 deg)
wrist_2_joint	-1.570796 rad (-90 deg)
wrist_3_joint	-1.570796 rad (-90 deg)

Ur20 (`ur20`)

ur20 ur20


DoF	6
End Effector	None
Base Frame	`base_link`
Flange Frame	`wrist_3_link`
Sensors	3
Action Modes	5 (default: `diff_ik`)

Action Modes

Mode	Controller	Dims	Scale
`diff_ik` *	Differential Inverse Kinematics	6	0.1
`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
`joint_pos_rel`	Relative Joint Position	6	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
arm	`.*`	87 N·m	100 rad/s	800	40

Sensors

Name	Type	Location
contacts_robot	Contact Sensor	`.*`
cam_base	Camera	`base_link/camera_base`
cam_wrist	Camera	`wrist_3_link/camera_wrist`

Initial Joint Configuration

Joint	Position
shoulder_pan_joint	0.0 rad
shoulder_lift_joint	-1.570796 rad (-90 deg)
elbow_joint	1.570796 rad (90 deg)
wrist_1_joint	-1.570796 rad (-90 deg)
wrist_2_joint	-1.570796 rad (-90 deg)
wrist_3_joint	-1.570796 rad (-90 deg)

Ur3 (`ur3`)

Compact collaborative robot arm from Universal Robots

ur3 ur3


DoF	6
End Effector	None
Base Frame	`base_link`
Flange Frame	`wrist_3_link`
Sensors	3
Action Modes	5 (default: `diff_ik`)
Source	Isaac Lab

Action Modes

Mode	Controller	Dims	Scale
`diff_ik` *	Differential Inverse Kinematics	6	0.1
`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
`joint_pos_rel`	Relative Joint Position	6	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
arm	`.*`	87 N·m	100 rad/s	800	40

Sensors

Name	Type	Location
contacts_robot	Contact Sensor	`.*`
cam_base	Camera	`base_link/camera_base`
cam_wrist	Camera	`wrist_3_link/camera_wrist`

Initial Joint Configuration

Joint	Position
shoulder_pan_joint	0.0 rad
shoulder_lift_joint	-1.570796 rad (-90 deg)
elbow_joint	1.570796 rad (90 deg)
wrist_1_joint	-1.570796 rad (-90 deg)
wrist_2_joint	-1.570796 rad (-90 deg)
wrist_3_joint	-1.570796 rad (-90 deg)

Ur30 (`ur30`)

ur30 ur30


DoF	6
End Effector	None
Base Frame	`base_link`
Flange Frame	`wrist_3_link`
Sensors	3
Action Modes	5 (default: `diff_ik`)

Action Modes

Mode	Controller	Dims	Scale
`diff_ik` *	Differential Inverse Kinematics	6	0.1
`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
`joint_pos_rel`	Relative Joint Position	6	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
arm	`.*`	87 N·m	100 rad/s	800	40

Sensors

Name	Type	Location
contacts_robot	Contact Sensor	`.*`
cam_base	Camera	`base_link/camera_base`
cam_wrist	Camera	`wrist_3_link/camera_wrist`

Initial Joint Configuration

Joint	Position
shoulder_pan_joint	0.0 rad
shoulder_lift_joint	-1.570796 rad (-90 deg)
elbow_joint	1.570796 rad (90 deg)
wrist_1_joint	-1.570796 rad (-90 deg)
wrist_2_joint	-1.570796 rad (-90 deg)
wrist_3_joint	-1.570796 rad (-90 deg)

Ur3E (`ur3e`)

ur3e ur3e


DoF	6
End Effector	None
Base Frame	`base_link`
Flange Frame	`wrist_3_link`
Sensors	3
Action Modes	5 (default: `diff_ik`)

Action Modes

Mode	Controller	Dims	Scale
`diff_ik` *	Differential Inverse Kinematics	6	0.1
`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
`joint_pos_rel`	Relative Joint Position	6	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
arm	`.*`	87 N·m	100 rad/s	800	40

Sensors

Name	Type	Location
contacts_robot	Contact Sensor	`.*`
cam_base	Camera	`base_link/camera_base`
cam_wrist	Camera	`wrist_3_link/camera_wrist`

Initial Joint Configuration

Joint	Position
shoulder_pan_joint	0.0 rad
shoulder_lift_joint	-1.570796 rad (-90 deg)
elbow_joint	1.570796 rad (90 deg)
wrist_1_joint	-1.570796 rad (-90 deg)
wrist_2_joint	-1.570796 rad (-90 deg)
wrist_3_joint	-1.570796 rad (-90 deg)

Ur5 (`ur5`)

Versatile mid-range collaborative arm from Universal Robots

ur5 ur5


DoF	6
End Effector	None
Base Frame	`base_link`
Flange Frame	`wrist_3_link`
Sensors	3
Action Modes	5 (default: `diff_ik`)
Source	Isaac Lab

Action Modes

Mode	Controller	Dims	Scale
`diff_ik` *	Differential Inverse Kinematics	6	0.1
`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
`joint_pos_rel`	Relative Joint Position	6	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
arm	`.*`	87 N·m	100 rad/s	800	40

Sensors

Name	Type	Location
contacts_robot	Contact Sensor	`.*`
cam_base	Camera	`base_link/camera_base`
cam_wrist	Camera	`wrist_3_link/camera_wrist`

Initial Joint Configuration

Joint	Position
shoulder_pan_joint	0.0 rad
shoulder_lift_joint	-1.570796 rad (-90 deg)
elbow_joint	1.570796 rad (90 deg)
wrist_1_joint	-1.570796 rad (-90 deg)
wrist_2_joint	-1.570796 rad (-90 deg)
wrist_3_joint	-1.570796 rad (-90 deg)

Ur5E (`ur5e`)

ur5e ur5e


DoF	6
End Effector	None
Base Frame	`base_link`
Flange Frame	`wrist_3_link`
Sensors	3
Action Modes	5 (default: `diff_ik`)

Action Modes

Mode	Controller	Dims	Scale
`diff_ik` *	Differential Inverse Kinematics	6	0.1
`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
`joint_pos_rel`	Relative Joint Position	6	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
arm	`.*`	87 N·m	100 rad/s	800	40

Sensors

Name	Type	Location
contacts_robot	Contact Sensor	`.*`
cam_base	Camera	`base_link/camera_base`
cam_wrist	Camera	`wrist_3_link/camera_wrist`

Initial Joint Configuration

Joint	Position
shoulder_pan_joint	0.0 rad
shoulder_lift_joint	-1.570796 rad (-90 deg)
elbow_joint	1.570796 rad (90 deg)
wrist_1_joint	-1.570796 rad (-90 deg)
wrist_2_joint	-1.570796 rad (-90 deg)
wrist_3_joint	-1.570796 rad (-90 deg)

SO-ARM100

So Arm100 D5 (`so_arm100_d5`)

5-DoF low-cost open-source motor-driven arm

so_arm100_d5


DoF	5
End Effector	SOArm100Gripper
Base Frame	`Base`
Flange Frame	`Fixed_Gripper`
Sensors	3
Action Modes	3 (default: `diff_ik`)
Source	SRB

Action Modes

Mode	Controller	Dims	Scale
`diff_ik` *	Differential Inverse Kinematics	6	0.05
`osc_var_stiffness`	Operational Space Controller	12	pos=0.05, ori=0.05, stiffness=120.0
`joint_pos_rel`	Relative Joint Position	5	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
arm	`.*`	10 N·m	10 rad/s	100	20

Sensors

Name	Type	Location
contacts_robot	Contact Sensor	`.*`
cam_base	Camera	`Base/camera_base`
cam_wrist	Camera	`Fixed_Gripper/camera_wrist`

Initial Joint Configuration

Joint	Position
Shoulder_Rotation	0.0 rad
Shoulder_Pitch	-0.261799 rad (-15 deg)
Elbow	0.349066 rad (20 deg)
Wrist_Pitch	0.087266 rad (5 deg)
Wrist_Roll	0.0 rad

So Arm100 D7 (`so_arm100_d7`)

7-DoF low-cost open-source motor-driven arm

so_arm100_d7


DoF	7
End Effector	None
Base Frame	`Base`
Flange Frame	`End_Servo`
Sensors	3
Action Modes	2 (default: `diff_ik`)
Source	SRB

Action Modes

Mode	Controller	Dims	Scale
`diff_ik` *	Differential Inverse Kinematics	6	0.05
`joint_pos_rel`	Relative Joint Position	7	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
arm	`.*`	10 N·m	10 rad/s	100	20

Sensors

Name	Type	Location
contacts_robot	Contact Sensor	`.*`
cam_base	Camera	`Base/camera_base`
cam_wrist	Camera	`End_Servo/camera_wrist`

Initial Joint Configuration

Joint	Position
Shoulder_Pitch	2.094395 rad (120 deg)
Shoulder_Yaw	0.0 rad
Humeral_Rotation	0.0 rad
Elbow	-1.570796 rad (-90 deg)
Wrist_Roll	0.0 rad
Wrist_Yaw	0.0 rad
Wrist_Pitch	-1.570796 rad (-90 deg)

Space

Canadarm3 (`canadarm3`)

Large-scale space manipulator for the Lunar Gateway station

canadarm3


DoF	7
End Effector	None
Base Frame	`canadarm3_large_0`
Flange Frame	`canadarm3_large_7`
Sensors	3
Action Modes	2 (default: `diff_ik`)
Source	SRB

Action Modes

Mode	Controller	Dims	Scale
`diff_ik` *	Differential Inverse Kinematics	6	0.1
`joint_pos_rel`	Relative Joint Position	7	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
joints	`canadarm3_large_joint_[1-7]`	2500 N·m	5 rad/s	40000	25000

Sensors

Name	Type	Location
contacts_robot	Contact Sensor	`.*`
cam_base	Camera	`canadarm3_large_0/camera_base`
cam_wrist	Camera	`canadarm3_large_7/camera_wrist`

Initial Joint Configuration

Joint	Position
canadarm3_large_joint_1	0.872665 rad (50 deg)
canadarm3_large_joint_2	0.0 rad
canadarm3_large_joint_3	0.959931 rad (55 deg)
canadarm3_large_joint_4	1.308997 rad (75 deg)
canadarm3_large_joint_5	-0.523599 rad (-30 deg)
canadarm3_large_joint_6	0.0 rad
canadarm3_large_joint_7	0.0 rad

Vispa (`vispa`)

Space manipulator designed for planetary surface operations

vispa vispa


DoF	6
End Effector	None
Base Frame	`link0`
Flange Frame	`link6`
Sensors	3
Action Modes	3 (default: `diff_ik`)
Source	SRB

Action Modes

Mode	Controller	Dims	Scale
`diff_ik` *	Differential Inverse Kinematics	6	0.05
`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
`joint_pos_rel`	Relative Joint Position	6	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
joints	`joint[1-6]`	500 N·m	5 rad/s	2500	500

Sensors

Name	Type	Location
contacts_robot	Contact Sensor	`.*`
cam_base	Camera	`link0/camera_base`
cam_wrist	Camera	`link6/camera_wrist`

Initial Joint Configuration

Joint	Position
joint1	0.0 rad
joint2	-0.523599 rad (-30 deg)
joint3	2.094395 rad (120 deg)
joint4	0.0 rad
joint5	1.570796 rad (90 deg)
joint6	0.0 rad

Unitree

Unitree Z1 (`unitree_z1`)

Compact lightweight arm from Unitree Robotics

unitree_z1


DoF	6
End Effector	None
Base Frame	`link00`
Flange Frame	`link06`
Sensors	3
Action Modes	2 (default: `diff_ik`)
Source	SRB

Action Modes

Mode	Controller	Dims	Scale
`diff_ik` *	Differential Inverse Kinematics	6	0.05
`joint_pos_rel`	Relative Joint Position	6	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
arm	`.*`	87 N·m	100 rad/s	800	40

Sensors

Name	Type	Location
contacts_robot	Contact Sensor	`.*`
cam_base	Camera	`link00/camera_base`
cam_wrist	Camera	`link06/camera_wrist`

Initial Joint Configuration

Joint	Position
joint1	0.0 rad
joint2	1.570796 rad (90 deg)
joint3	-0.785398 rad (-45 deg)
joint4	0.785398 rad (45 deg)
joint5	0.0 rad
joint6	0.0 rad

Tools

Grippers, hands, and other end-of-arm tools registered in the Space Robotics Bench. These attach to robot arms and enable dexterous manipulation, sample collection, and assembly operations.

Overview

Active

Allegro Hand (`allegro_hand`)

allegro_hand


Type	tool
Variant	Dataset
Domains	–
Collision	–

Electric Screwdriver M3 (`electric_screwdriver_m3`)

electric_screwdriver_m3


Type	tool
Variant	Dataset
Domains	–
Collision	–

Electric Screwdriver M5 (`electric_screwdriver_m5`)

electric_screwdriver_m5


Type	tool
Variant	Dataset
Domains	–
Collision	–

Franka Hand (`franka_hand`)

franka_hand


Type	tool
Variant	Dataset
Domains	–
Collision	–

Kinova300 (`kinova300`)

kinova300


Type	tool
Variant	Dataset
Domains	–
Collision	–

Robotiq Hand E (`robotiq_hand_e`)

robotiq_hand_e


Type	tool
Variant	Dataset
Domains	–
Collision	–

Shadow Hand (`shadow_hand`)

shadow_hand


Type	tool
Variant	Dataset
Domains	–
Collision	–

So Arm100 Gripper (`so_arm100_gripper`)

so_arm100_gripper


Type	tool
Variant	Dataset
Domains	–
Collision	–

Passive

Manual Screwdriver M3 (`manual_screwdriver_m3`)

manual_screwdriver_m3


Type	tool
Variant	Dataset
Domains	–
Collision	sdf
Density	1500 kg/m³

Manual Screwdriver M5 (`manual_screwdriver_m5`)

manual_screwdriver_m5


Type	tool
Variant	Dataset
Domains	–
Collision	sdf
Density	1500 kg/m³

Random Scoop (`random_scoop`)

random_scoop


Type	tool
Variant	Dataset
Domains	–
Collision	–

Scoop (`scoop`)

scoop scoop


Type	tool
Variant	Procedural
Domains	–
Collision	sdf

Scoop Custom1 (`scoop_custom1`)

scoop_custom1


Type	tool
Variant	Dataset
Domains	–
Collision	sdf

Scoop Custom2 (`scoop_custom2`)

scoop_custom2


Type	tool
Variant	Dataset
Domains	–
Collision	sdf

Scoop Custom3 (`scoop_custom3`)

scoop_custom3


Type	tool
Variant	Dataset
Domains	–
Collision	sdf

Scoop Rectangular (`scoop_rectangular`)

scoop_rectangular


Type	tool
Variant	Dataset
Domains	–
Collision	sdf

Scoop Spherical (`scoop_spherical`)

scoop_spherical


Type	tool
Variant	Dataset
Domains	–
Collision	sdf

Scoop Triangular (`scoop_triangular`)

scoop_triangular


Type	tool
Variant	Dataset
Domains	–
Collision	sdf

Mobile Robots

Mobile platforms registered in the Space Robotics Bench. Includes wheeled rovers, legged platforms, and aerial vehicles for navigation and locomotion tasks.

Overview

Landers

Apollo Lander (`apollo_lander`)

apollo_lander


DoF	runtime-resolved
End Effector	None
Base Frame	`base`
Flange Frame	–
Sensors	2
Action Modes	default: `thrust`

Action Modes

Mode	Controller	Dims	Scale
`thrust` *	Thrust	17	1.0

Sensors

Name	Type	Location
imu_robot	Imu	`base`
cam_onboard	Camera	`base/camera_onboard`

Peregrine Lander (`peregrine_lander`)

peregrine_lander


DoF	runtime-resolved
End Effector	None
Base Frame	`base`
Flange Frame	–
Sensors	2
Action Modes	default: `thrust`

Action Modes

Mode	Controller	Dims	Scale
`thrust` *	Thrust	17	1.0

Sensors

Name	Type	Location
imu_robot	Imu	`base`
cam_onboard	Camera	`base/camera_onboard`

Random Lander (`random_lander`)

random_lander


DoF	runtime-resolved
End Effector	None
Base Frame	`base`
Flange Frame	–
Sensors	2
Action Modes	default: `thrust`

Action Modes

Mode	Controller	Dims	Scale
`thrust` *	Thrust	3	1.0

Sensors

Name	Type	Location
imu_robot	Imu	`base`
cam_onboard	Camera	`base/camera_onboard`

Resilience Lander (`resilience_lander`)

resilience_lander


DoF	runtime-resolved
End Effector	None
Base Frame	`base`
Flange Frame	–
Sensors	2
Action Modes	default: `thrust`

Action Modes

Mode	Controller	Dims	Scale
`thrust` *	Thrust	9	1.0

Sensors

Name	Type	Location
imu_robot	Imu	`base`
cam_onboard	Camera	`base/camera_onboard`

Vikram Lander (`vikram_lander`)

vikram_lander


DoF	runtime-resolved
End Effector	None
Base Frame	`base`
Flange Frame	–
Sensors	2
Action Modes	default: `thrust`

Action Modes

Mode	Controller	Dims	Scale
`thrust` *	Thrust	13	1.0

Sensors

Name	Type	Location
imu_robot	Imu	`base`
cam_onboard	Camera	`base/camera_onboard`

Legged Robots

Anymal C (`anymal_c`)

anymal_c


DoF	runtime-resolved
End Effector	None
Base Frame	`base`
Flange Frame	–
Sensors	3
Action Modes	default: `joint_pos`

Action Modes

Mode	Controller	Dims	Scale
`joint_pos` *	Joint Position To Limits	runtime-resolved	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
legs	`.HAA`, `.HFE`, `.*KFE`	80 N·m	7.5 rad/s	–	–

Sensors

Name	Type	Location
imu_robot	Imu	`base`
contacts_feet	Contact Sensor	`.*FOOT`
cam_front	Camera	`base/camera_front`

Anymal D (`anymal_d`)

anymal_d


DoF	runtime-resolved
End Effector	None
Base Frame	`base`
Flange Frame	–
Sensors	3
Action Modes	default: `joint_pos`

Action Modes

Mode	Controller	Dims	Scale
`joint_pos` *	Joint Position To Limits	runtime-resolved	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
legs	`.HAA`, `.HFE`, `.*KFE`	80 N·m	7.5 rad/s	–	–

Sensors

Name	Type	Location
imu_robot	Imu	`base`
contacts_feet	Contact Sensor	`.*FOOT`
cam_front	Camera	`base/camera_front`

Cassie (`cassie`)

cassie cassie


DoF	12
End Effector	None
Base Frame	`pelvis`
Flange Frame	–
Sensors	3
Action Modes	default: `joint_pos`

Action Modes

Mode	Controller	Dims	Scale
`joint_pos` *	Joint Position To Limits	12	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
legs	`hip_.`, `thigh_.`, `ankle_.*`	200 N·m	10 rad/s	100	3
toes	`toe_.*`	20 N·m	10 rad/s	20	1

Sensors

Name	Type	Location
imu_robot	Imu	`pelvis`
contacts_feet	Contact Sensor	`(left\|right)_toe`
cam_front	Camera	`pelvis/camera_front`

Initial Joint Configuration

Joint	Position
hip_abduction_left	0.1 rad (5.729578 deg)
hip_rotation_left	0.0 rad
hip_flexion_left	1 rad (57.29578 deg)
thigh_joint_left	-1.8 rad (-103.132403 deg)
ankle_joint_left	1.57 rad (89.954374 deg)
toe_joint_left	-1.57 rad (-89.954374 deg)
hip_abduction_right	-0.1 rad (-5.729578 deg)
hip_rotation_right	0.0 rad
hip_flexion_right	1 rad (57.29578 deg)
thigh_joint_right	-1.8 rad (-103.132403 deg)
ankle_joint_right	1.57 rad (89.954374 deg)
toe_joint_right	-1.57 rad (-89.954374 deg)

Random Anymal Quadruped (`random_anymal_quadruped`)

random_anymal_quadruped


DoF	runtime-resolved
End Effector	None
Base Frame	`base`
Flange Frame	–
Sensors	3
Action Modes	default: `joint_pos`

Action Modes

Mode	Controller	Dims	Scale
`joint_pos` *	Joint Position To Limits	runtime-resolved	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
legs	`.HAA`, `.HFE`, `.*KFE`	80 N·m	7.5 rad/s	–	–

Sensors

Name	Type	Location
imu_robot	Imu	`base`
contacts_feet	Contact Sensor	`.*FOOT`
cam_front	Camera	`base/camera_front`

Random Unitree Quadruped (`random_unitree_quadruped`)

random_unitree_quadruped


DoF	runtime-resolved
End Effector	None
Base Frame	`trunk`
Flange Frame	–
Sensors	3
Action Modes	default: `joint_pos`

Action Modes

Mode	Controller	Dims	Scale
`joint_pos` *	Joint Position To Limits	runtime-resolved	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
base_legs	`._hip_joint`, `._thigh_joint`, `.*_calf_joint`	33.5 N·m	21 rad/s	25	0.5

Sensors

Name	Type	Location
imu_robot	Imu	`trunk`
contacts_feet	Contact Sensor	`.*foot`
cam_front	Camera	`trunk/camera_front`

Initial Joint Configuration

Joint	Position
F[L,R]_thigh_joint	0.8 rad (45.836624 deg)
R[L,R]_thigh_joint	1 rad (57.29578 deg)

Spot (`spot`)

spot spot


DoF	runtime-resolved
End Effector	None
Base Frame	`body`
Flange Frame	–
Sensors	3
Action Modes	default: `joint_pos`

Action Modes

Mode	Controller	Dims	Scale
`joint_pos` *	Joint Position	runtime-resolved	0.2

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
spot_hip	`.*_h[xy]`	45 N·m	–	60	1.5
spot_knee	`.*_kn`	–	–	60	1.5

Sensors

Name	Type	Location
imu_robot	Imu	`body`
contacts_feet	Contact Sensor	`.*foot`
cam_front	Camera	`body/camera_front`

Initial Joint Configuration

Joint	Position
[fh]l_hx	0.1 rad (5.729578 deg)
[fh]r_hx	-0.1 rad (-5.729578 deg)
f[rl]_hy	0.9 rad (51.566202 deg)
h[rl]_hy	1.1 rad (63.025357 deg)

Unitree A1 (`unitree_a1`)

unitree_a1


DoF	runtime-resolved
End Effector	None
Base Frame	`trunk`
Flange Frame	–
Sensors	3
Action Modes	default: `joint_pos`

Action Modes

Mode	Controller	Dims	Scale
`joint_pos` *	Joint Position To Limits	runtime-resolved	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
base_legs	`._hip_joint`, `._thigh_joint`, `.*_calf_joint`	33.5 N·m	21 rad/s	25	0.5

Sensors

Name	Type	Location
imu_robot	Imu	`trunk`
contacts_feet	Contact Sensor	`.*foot`
cam_front	Camera	`trunk/camera_front`

Initial Joint Configuration

Joint	Position
F[L,R]_thigh_joint	0.8 rad (45.836624 deg)
R[L,R]_thigh_joint	1 rad (57.29578 deg)

Unitree Go1 (`unitree_go1`)

unitree_go1


DoF	runtime-resolved
End Effector	None
Base Frame	`trunk`
Flange Frame	–
Sensors	3
Action Modes	default: `joint_pos`

Action Modes

Mode	Controller	Dims	Scale
`joint_pos` *	Joint Position To Limits	runtime-resolved	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
base_legs	`._hip_joint`, `._thigh_joint`, `.*_calf_joint`	23.7 N·m	30 rad/s	–	–

Sensors

Name	Type	Location
imu_robot	Imu	`trunk`
contacts_feet	Contact Sensor	`.*foot`
cam_front	Camera	`trunk/camera_front`

Initial Joint Configuration

Joint	Position
F[L,R]_thigh_joint	0.8 rad (45.836624 deg)
R[L,R]_thigh_joint	1 rad (57.29578 deg)

Unitree Go2 (`unitree_go2`)

unitree_go2


DoF	runtime-resolved
End Effector	None
Base Frame	`base`
Flange Frame	–
Sensors	3
Action Modes	default: `joint_pos`

Action Modes

Mode	Controller	Dims	Scale
`joint_pos` *	Joint Position To Limits	runtime-resolved	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
base_legs	`._hip_joint`, `._thigh_joint`, `.*_calf_joint`	33.5 N·m	21 rad/s	25	0.5

Sensors

Name	Type	Location
imu_robot	Imu	`base`
contacts_feet	Contact Sensor	`.*foot`
cam_front	Camera	`base/camera_front`

Initial Joint Configuration

Joint	Position
F[L,R]_thigh_joint	0.8 rad (45.836624 deg)
R[L,R]_thigh_joint	1 rad (57.29578 deg)

Multicopters

Crazyflie (`crazyflie`)

crazyflie


DoF	runtime-resolved
End Effector	None
Base Frame	`body`
Flange Frame	–
Sensors	2
Action Modes	default: `body_acc`

Action Modes

Mode	Controller	Dims	Scale
`body_acc` *	Multicopter Body Acceleration	runtime-resolved	0.5

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
dummy	`.*`	–	–	0	0

Sensors

Name	Type	Location
imu_robot	Imu	`body`
cam_onboard	Camera	`body/camera_onboard`

Ingenuity (`ingenuity`)

ingenuity


DoF	2
End Effector	None
Base Frame	`body`
Flange Frame	–
Sensors	3
Action Modes	default: `body_acc`

Action Modes

Mode	Controller	Dims	Scale
`body_acc` *	Multicopter Body Acceleration	2	0.5

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
rotors	`rotor_joint_[1-2]`	7.5 N·m	261.799388 rad/s	0	1000

Sensors

Name	Type	Location
imu_robot	Imu	`body`
cam_onboard	Camera	`body/camera_onboard`
altimeter	Heightmap Sensor	`body`

Orbital Robots

Cubesat (`cubesat`)

cubesat cubesat


DoF	runtime-resolved
End Effector	None
Base Frame	`cubesat`
Flange Frame	–
Sensors	2
Action Modes	default: `thrust`

Action Modes

Mode	Controller	Dims	Scale
`thrust` *	Thrust	6	1.0

Sensors

Name	Type	Location
imu_robot	Imu	`cubesat`
cam_onboard	Camera	`cubesat/camera_onboard`

Floating Platform (`floating_platform`)

floating_platform


DoF	1
End Effector	None
Base Frame	`base_link`
Flange Frame	–
Sensors	2
Action Modes	2 (default: `thrust_only`)

Action Modes

Mode	Controller	Dims	Scale
`thrust_only` *	Thrust	8	1.0
`thrust_and_reaction`	Thrust	9	thrust=1.0, reaction_wheel_torque=0.05

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
reaction_wheel	`rw_revolute_joint`	1 N·m	100 rad/s	0	0

Sensors

Name	Type	Location
imu_robot	Imu	`base_link`
cam_onboard	Camera	`base_link/camera_onboard`

Gateway (`gateway`)

gateway gateway


DoF	runtime-resolved
End Effector	None
Base Frame	`gateway`
Flange Frame	–
Sensors	2
Action Modes	default: `body_acc`

Action Modes

Mode	Controller	Dims	Scale
`body_acc` *	Body Acceleration	runtime-resolved	0.05

Sensors

Name	Type	Location
imu_robot	Imu	`gateway`
cam_onboard	Camera	`gateway/camera_onboard`

Int Ball1 (`int_ball1`)

int_ball1


DoF	runtime-resolved
End Effector	None
Base Frame	`base_link`
Flange Frame	–
Sensors	2
Action Modes	default: `thrust`

Action Modes

Mode	Controller	Dims	Scale
`thrust` *	Thrust	6	1.0

Sensors

Name	Type	Location
imu_robot	Imu	`base_link`
cam_onboard	Camera	`base_link/camera_onboard`

Int Ball2 (`int_ball2`)

int_ball2


DoF	runtime-resolved
End Effector	None
Base Frame	`base_link`
Flange Frame	–
Sensors	2
Action Modes	default: `thrust`

Action Modes

Mode	Controller	Dims	Scale
`thrust` *	Thrust	6	1.0

Sensors

Name	Type	Location
imu_robot	Imu	`base_link`
cam_onboard	Camera	`base_link/camera_onboard`

Iss (`iss`)

iss iss


DoF	runtime-resolved
End Effector	None
Base Frame	`base`
Flange Frame	–
Sensors	2
Action Modes	default: `body_acc`

Action Modes

Mode	Controller	Dims	Scale
`body_acc` *	Body Acceleration	runtime-resolved	0.05

Sensors

Name	Type	Location
imu_robot	Imu	`base`
cam_onboard	Camera	`base/camera_onboard`

Satellite Mockup (`satellite_mockup`)

satellite_mockup


DoF	runtime-resolved
End Effector	None
Base Frame	`satellite_mockup`
Flange Frame	–
Sensors	2
Action Modes	default: `thrust`

Action Modes

Mode	Controller	Dims	Scale
`thrust` *	Thrust	12	1.0

Sensors

Name	Type	Location
imu_robot	Imu	`satellite_mockup`
cam_onboard	Camera	`satellite_mockup/camera_onboard`

Starship (`starship`)

starship


DoF	runtime-resolved
End Effector	None
Base Frame	`base`
Flange Frame	–
Sensors	2
Action Modes	default: `thrust`

Action Modes

Mode	Controller	Dims	Scale
`thrust` *	Thrust	18	1.0

Sensors

Name	Type	Location
imu_robot	Imu	`base`
cam_onboard	Camera	`base/camera_onboard`

Super Heavy (`super_heavy`)

super_heavy


DoF	runtime-resolved
End Effector	None
Base Frame	`base`
Flange Frame	–
Sensors	2
Action Modes	default: `thrust`

Action Modes

Mode	Controller	Dims	Scale
`thrust` *	Thrust	59	1.0

Sensors

Name	Type	Location
imu_robot	Imu	`base`
cam_onboard	Camera	`base/camera_onboard`

Venus Express (`venus_express`)

venus_express


DoF	runtime-resolved
End Effector	None
Base Frame	`base`
Flange Frame	–
Sensors	2
Action Modes	default: `thrust`

Action Modes

Mode	Controller	Dims	Scale
`thrust` *	Thrust	19	1.0

Sensors

Name	Type	Location
imu_robot	Imu	`base`
cam_onboard	Camera	`base/camera_onboard`

Surface Vehicles

Heron (`heron`)

heron heron


DoF	runtime-resolved
End Effector	None
Base Frame	`base_link`
Flange Frame	–
Sensors	2
Action Modes	default: `thrust`

Action Modes

Mode	Controller	Dims	Scale
`thrust` *	Thrust	2	1.0

Sensors

Name	Type	Location
imu_robot	Imu	`base_link`
cam_onboard	Camera	`base_link/camera_onboard`

Wheeled Robots

Cadre (`cadre`)

cadre cadre


DoF	runtime-resolved
End Effector	None
Base Frame	`chassis`
Flange Frame	–
Sensors	3
Action Modes	default: `drive`

Action Modes

Mode	Controller	Dims	Scale
`drive` *	Wheeled Drive	runtime-resolved	lin=0.4, ang=0.7853981633974483

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
wheel_joints	`wheel_joint_.*`	250 N·m	6.28 rad/s	0	2000
suspension_joints	`suspension_joint.*`	500 N·m	0.25 rad/s	400	5

Sensors

Name	Type	Location
imu_robot	Imu	`chassis`
contacts_wheels	Contact Sensor	`.wheel.`
cam_front	Camera	`chassis/camera_front`

Husky (`husky`)

husky husky


DoF	runtime-resolved
End Effector	None
Base Frame	`chassis`
Flange Frame	–
Sensors	3
Action Modes	default: `drive`

Action Modes

Mode	Controller	Dims	Scale
`drive` *	Wheeled Drive	runtime-resolved	lin=0.1, ang=0.7853981633974483

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
wheel_joints	`wheel_joint_.*`	250 N·m	6.28 rad/s	0	2000

Sensors

Name	Type	Location
imu_robot	Imu	`chassis`
contacts_wheels	Contact Sensor	`.wheel.`
cam_front	Camera	`chassis/camera_front`

Leo Rover (`leo_rover`)

leo_rover


DoF	runtime-resolved
End Effector	None
Base Frame	`chassis`
Flange Frame	–
Sensors	3
Action Modes	default: `drive`

Action Modes

Mode	Controller	Dims	Scale
`drive` *	Wheeled Drive	runtime-resolved	lin=0.4, ang=1.0471975511965976

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
drive	`wheel_joint_.*`	80 N·m	30 rad/s	0	5000
rocker	`rocker_joint_.*`	500 N·m	2 rad/s	2	0.5

Sensors

Name	Type	Location
imu_robot	Imu	`chassis`
contacts_wheels	Contact Sensor	`.wheel.`
cam_front	Camera	`chassis/camera_front`

Mars Rover (`mars_rover`)

mars_rover


DoF	runtime-resolved
End Effector	None
Base Frame	`Body`
Flange Frame	–
Sensors	3
Action Modes	default: `drive`

Action Modes

Mode	Controller	Dims	Scale
`drive` *	Wheeled Drive	runtime-resolved	lin=0.5, ang=0.7853981633974483

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
base_steering	`.Steer.`	12 N·m	6 rad/s	8000	1000
base_drive	`.Drive.`	12 N·m	6 rad/s	100	4000
passive_joints_boogie	`.*RevoluteJoint`	–	15 rad/s	0	0
passive_joints_rocker	`.Rocker.`	–	15 rad/s	0	0
passive_joints_differential	`.Differential.`	–	15 rad/s	0	0

Sensors

Name	Type	Location
imu_robot	Imu	`Body`
contacts_wheels	Contact Sensor	`.[Dd]rive.`
cam_front	Camera	`Body/camera_front`

Nifty (`nifty`)

nifty nifty


DoF	12
End Effector	None
Base Frame	`chassis`
Flange Frame	–
Sensors	3
Action Modes	default: `nifty`

Action Modes

Mode	Controller	Dims	Scale
`nifty` *	Joint Velocity	12	joint_vel_wheel_drive=1.5707963267948966, joint_pos_steer=0.7853981633974483, joint_pos_suspension_yaw=0.5235987755982988, joint_pos_suspension_roll=0.2617993877991494

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
wheel_joints	`wheel_joint_fl`, `wheel_joint_fr`, `wheel_joint_rl`, `wheel_joint_rr`	80 N·m	1.570796 rad/s	0	80
steer_joints	`steer_joint_fl`, `steer_joint_fr`, `steer_joint_rl`, `steer_joint_rr`	200 N·m	0.523599 rad/s	100	50
suspension_joint_yaw	`suspension_joint_yaw_f`, `suspension_joint_yaw_r`	200 N·m	0.261799 rad/s	1250	750
suspension_joint_roll	`suspension_joint_roll_f`, `suspension_joint_roll_r`	200 N·m	0.261799 rad/s	15000	5000

Sensors

Name	Type	Location
imu_robot	Imu	`chassis`
contacts_wheels	Contact Sensor	`.wheel.`
cam_front	Camera	`chassis`

Open Source Rover (`open_source_rover`)

open_source_rover


DoF	runtime-resolved
End Effector	None
Base Frame	`chassis`
Flange Frame	–
Sensors	3
Action Modes	default: `drive`

Action Modes

Mode	Controller	Dims	Scale
`drive` *	Wheeled Drive	runtime-resolved	lin=0.2, ang=0.3490658503988659

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
joint_wheel	`joint_wheel_.*`	50 N·m	40 rad/s	0	2000
joints_steer	`joint_steer_.*`	500 N·m	0.5 rad/s	100	80
joints_rocker	`joint_rocker_.*`	1500 N·m	0.3 rad/s	25	2
joints_bogie	`joint_bogie_.*`	750 N·m	0.6 rad/s	1	4
joint_linkage	`joint_linkage`	25 N·m	0.2 rad/s	0.05	0.1

Sensors

Name	Type	Location
imu_robot	Imu	`chassis`
contacts_wheels	Contact Sensor	`.wheel.`
cam_front	Camera	`chassis/camera_front`

Perseverance (`perseverance`)

perseverance


DoF	runtime-resolved
End Effector	None
Base Frame	`body`
Flange Frame	–
Sensors	3
Action Modes	default: `drive`

Action Modes

Mode	Controller	Dims	Scale
`drive` *	Wheeled Drive	runtime-resolved	lin=0.7, ang=1.3089969389957472

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
drive_joints	`drive_joint.*`	150 N·m	40 rad/s	0	25000
steer_joints	`steer_joint.*`	400 N·m	2 rad/s	500	200
rocker_joints	`suspension_joint_rocker.*`	2500 N·m	5 rad/s	4000	400
bogie_joints	`suspension_joint_bogie.*`	500 N·m	4 rad/s	200	25

Sensors

Name	Type	Location
imu_robot	Imu	`body`
contacts_wheels	Contact Sensor	`.wheel.`
cam_front	Camera	`body/camera_front`

Pragyan (`pragyan`)

pragyan pragyan


DoF	runtime-resolved
End Effector	None
Base Frame	`body`
Flange Frame	–
Sensors	3
Action Modes	default: `drive`

Action Modes

Mode	Controller	Dims	Scale
`drive` *	Wheeled Drive	runtime-resolved	lin=0.35, ang=1.3962634015954636

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
drive_joints	`wheel_drive_joint_.*`	150 N·m	40 rad/s	0	5000
rocker_joints	`rocker_joint_.*`	2500 N·m	5 rad/s	1000	400
bogie_joints	`boogie_joint_.*`	500 N·m	4 rad/s	250	200

Sensors

Name	Type	Location
imu_robot	Imu	`body`
contacts_wheels	Contact Sensor	`.wheel.`
cam_front	Camera	`body/camera_front`

Raph Rover (`raph_rover`)


DoF	runtime-resolved
End Effector	None
Base Frame	`chassis`
Flange Frame	–
Sensors	3
Action Modes	2 (default: `drive`)

Action Modes

Mode	Controller	Dims	Scale
`drive` *	Wheeled Drive	runtime-resolved	lin=0.4, ang=1.0471975511965976
`drive_filtered`	Filtered Wheeled Drive	runtime-resolved	lin=0.4, ang=1.0471975511965976

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
drive	`drive_joint_.*`	80 N·m	30 rad/s	0	500
steer	`steer_joint_.*`	500 N·m	2 rad/s	400	50

Sensors

Name	Type	Location
imu_robot	Imu	`chassis`
contacts_wheels	Contact Sensor	`wheel.*`
cam_front	Camera	`chassis/camera_front`

Mobile Manipulators

Legged and wheeled platforms with integrated manipulation capabilities, including humanoid robots, registered in the Space Robotics Bench.

Overview

Combined Platforms

Generic Aerial Manipulator (`generic_aerial_manipulator`)


DoF	runtime-resolved
End Effector	None
Base Frame	–
Flange Frame	–
Sensors	0
Action Modes	–

Generic Ground Manipulator (`generic_ground_manipulator`)


DoF	runtime-resolved
End Effector	None
Base Frame	–
Flange Frame	–
Sensors	0
Action Modes	–

Generic Orbital Manipulator (`generic_orbital_manipulator`)


DoF	runtime-resolved
End Effector	None
Base Frame	–
Flange Frame	–
Sensors	0
Action Modes	–

Humanoids

Humanoid21 (`humanoid21`)

humanoid21


DoF	runtime-resolved
End Effector	None
Base Frame	`pelvis`
Flange Frame	–
Sensors	3
Action Modes	default: `joint_pos`

Action Modes

Mode	Controller	Dims	Scale
`joint_pos` *	Joint Position To Limits	runtime-resolved	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
body	`.*`	–	–	20	5

Sensors

Name	Type	Location
imu_robot	Imu	`pelvis`
contacts_feet	Contact Sensor	`(left\|right)_foot`
cam_front	Camera	`pelvis/camera_front`

Humanoid28 (`humanoid28`)

humanoid28


DoF	runtime-resolved
End Effector	None
Base Frame	`pelvis`
Flange Frame	–
Sensors	3
Action Modes	default: `joint_pos`

Action Modes

Mode	Controller	Dims	Scale
`joint_pos` *	Joint Position To Limits	runtime-resolved	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
body	`.*`	–	–	–	–

Sensors

Name	Type	Location
imu_robot	Imu	`pelvis`
contacts_feet	Contact Sensor	`(left\|right)_foot`
cam_front	Camera	`pelvis/camera_front`

Unitree G1 (`unitree_g1`)

unitree_g1


DoF	runtime-resolved
End Effector	None
Base Frame	`pelvis`
Flange Frame	–
Sensors	3
Action Modes	default: `joint_pos`

Action Modes

Mode	Controller	Dims	Scale
`joint_pos` *	Joint Position To Limits	runtime-resolved	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
legs	`._hip_yaw_joint`, `._hip_roll_joint`, `._hip_pitch_joint`, `._knee_joint`, `torso_joint`	300 N·m	100 rad/s	150	5
feet	`._ankle_pitch_joint`, `._ankle_roll_joint`	20 N·m	–	20	2
arms	`._shoulder_pitch_joint`, `._shoulder_roll_joint`, `._shoulder_yaw_joint`, `._elbow_pitch_joint`, `._elbow_roll_joint`, `._five_joint`, `._three_joint`, `._six_joint`, `._four_joint`, `._zero_joint`, `._one_joint`, `._two_joint`	300 N·m	100 rad/s	40	10

Sensors

Name	Type	Location
imu_robot	Imu	`pelvis`
contacts_feet	Contact Sensor	`(left\|right)_ankle_roll_link`
cam_front	Camera	`pelvis/camera_front`

Initial Joint Configuration

Joint	Position
left_shoulder_roll_joint	0.16 rad (9.167325 deg)
left_shoulder_pitch_joint	0.35 rad (20.053523 deg)
right_shoulder_roll_joint	-0.16 rad (-9.167325 deg)
right_shoulder_pitch_joint	0.35 rad (20.053523 deg)
left_one_joint	1 rad (57.29578 deg)
right_one_joint	-1 rad (-57.29578 deg)
left_two_joint	0.52 rad (29.793805 deg)
right_two_joint	-0.52 rad (-29.793805 deg)

The DoF and joint_pos action dimensions are runtime-resolved from the Unitree G1 USD articulation. The actuator configuration includes shoulder, elbow, ankle, and hand-like zero through six joint patterns.

Unitree H1 (`unitree_h1`)

unitree_h1


DoF	runtime-resolved
End Effector	None
Base Frame	`pelvis`
Flange Frame	–
Sensors	3
Action Modes	default: `joint_pos`

Action Modes

Mode	Controller	Dims	Scale
`joint_pos` *	Joint Position To Limits	runtime-resolved	1.0

Actuators

Group	Joints	Effort	Velocity	Stiffness	Damping
legs	`._hip_yaw`, `._hip_roll`, `._hip_pitch`, `._knee`, `torso`	300 N·m	100 rad/s	150	5
feet	`.*_ankle`	100 N·m	100 rad/s	20	4
arms	`._shoulder_pitch`, `._shoulder_roll`, `._shoulder_yaw`, `._elbow`	300 N·m	100 rad/s	40	10

Sensors

Name	Type	Location
imu_robot	Imu	`pelvis`
contacts_feet	Contact Sensor	`(left\|right)_ankle_link`
cam_front	Camera	`pelvis/camera_front`

Initial Joint Configuration

Joint	Position
torso	0.0 rad

Objects

Manipulable objects registered in the Space Robotics Bench. Includes pegs, bolts, samples, and other items used in manipulation tasks.

Overview

Construction Blocks

Lego Block 1X1 (`lego_block_1x1`)

lego_block_1x1


Type	modular_element
Variant	Dataset
Domains	–
Collision	sdf
Density	800 kg/m³

Lego Block 1X2 (`lego_block_1x2`)

lego_block_1x2


Type	modular_element
Variant	Dataset
Domains	–
Collision	sdf
Density	800 kg/m³

Lego Block 1X3 (`lego_block_1x3`)

lego_block_1x3


Type	modular_element
Variant	Dataset
Domains	–
Collision	sdf
Density	800 kg/m³

Lego Block 1X4 (`lego_block_1x4`)

lego_block_1x4


Type	modular_element
Variant	Dataset
Domains	–
Collision	sdf
Density	800 kg/m³

Lego Block 2X2 (`lego_block_2x2`)

lego_block_2x2


Type	modular_element
Variant	Dataset
Domains	–
Collision	sdf
Density	800 kg/m³

Lego Block 2X3 (`lego_block_2x3`)

lego_block_2x3


Type	modular_element
Variant	Dataset
Domains	–
Collision	sdf
Density	800 kg/m³

Lego Block 2X4 (`lego_block_2x4`)

lego_block_2x4


Type	modular_element
Variant	Dataset
Domains	–
Collision	sdf
Density	800 kg/m³

Random Lego Block (`random_lego_block`)

random_lego_block


Type	modular_element
Variant	Dataset
Domains	–
Collision	–

Geological Samples

Apollo Sample (`apollo_sample`)

apollo_sample


Type	common
Variant	Dataset
Domains	–
Collision	–

Asteroid (`asteroid`)

asteroid


Type	common
Variant	Procedural
Domains	–
Collision	convexHull
Density	2000 kg/m³

Lunalab Boulder (`lunalab_boulder`)

lunalab_boulder


Type	common
Variant	Dataset
Domains	–
Collision	–

Mars Rock (`mars_rock`)

mars_rock


Type	common
Variant	Procedural
Domains	–
Collision	convexHull
Density	2000 kg/m³

Moon Rock (`moon_rock`)

moon_rock


Type	common
Variant	Procedural
Domains	–
Collision	convexHull
Density	2000 kg/m³

Random Rock (`random_rock`)

random_rock


Type	common
Variant	Dataset
Domains	–
Collision	–
Density	2000 kg/m³

Sample Tube (`sample_tube`)

sample_tube


Type	common
Variant	Dataset
Domains	–
Collision	convexDecomposition
Density	1500 kg/m³

Spaceport Moon Rock (`spaceport_moon_rock`)

spaceport_moon_rock


Type	common
Variant	Dataset
Domains	–
Collision	–

Hardware

Bolt M8 (`bolt_m8`)

bolt_m8 bolt_m8


Type	common
Variant	Dataset
Domains	–
Collision	sdf

Bolt M8 Tight (`bolt_m8_tight`)

bolt_m8_tight


Type	common
Variant	Dataset
Domains	–
Collision	sdf

Hole (`hole`)

hole hole


Type	common
Variant	Procedural
Domains	–
Collision	–

Nut M8 (`nut_m8`)

nut_m8 nut_m8


Type	common
Variant	Dataset
Domains	–
Collision	–

Nut M8 Tight (`nut_m8_tight`)

nut_m8_tight


Type	common
Variant	Dataset
Domains	–
Collision	–

Peg (`peg`)

peg peg


Type	common
Variant	Procedural
Domains	–
Collision	sdf
Density	1000 kg/m³

Profile Hole (`profile_hole`)

profile_hole


Type	common
Variant	Dataset
Domains	–
Collision	–

Profile Peg (`profile_peg`)

profile_peg


Type	common
Variant	Dataset
Domains	–
Collision	boundingCube
Density	1000 kg/m³

Short Profile Peg (`short_profile_peg`)

short_profile_peg


Type	common
Variant	Dataset
Domains	–
Collision	boundingCube
Density	1000 kg/m³

Other

Beam (`beam`)


Type	common
Variant	Primitive
Domains	–
Collision	–
Density	250 kg/m³

Beneficiation Unit (`beneficiation_unit`)

beneficiation_unit


Type	common
Variant	Dataset
Domains	–
Collision	convexDecomposition
Density	1000 kg/m³

Cargo Bay (`cargo_bay`)

cargo_bay


Type	payload
Variant	Dataset
Domains	–
Collision	convexDecomposition
Density	1000 kg/m³

Husky Payload Lunalab (`husky_payload_lunalab`)

husky_payload_lunalab


Type	payload
Variant	Dataset
Domains	–
Collision	–
Density	500 kg/m³

Industrial Pedestal100 (`industrial_pedestal100`)

industrial_pedestal100


Type	pedestal
Variant	Dataset
Domains	–
Collision	–

Industrial Pedestal25 (`industrial_pedestal25`)

industrial_pedestal25


Type	pedestal
Variant	Dataset
Domains	–
Collision	–

Industrial Pedestal50 (`industrial_pedestal50`)

industrial_pedestal50


Type	pedestal
Variant	Dataset
Domains	–
Collision	–

Leo Shovel Rigid (`leo_shovel_rigid`)

leo_shovel_rigid


Type	payload
Variant	Dataset
Domains	–
Collision	sdf
Density	1500 kg/m³

Leo Shovel Tilt (`leo_shovel_tilt`)

leo_shovel_tilt


Type	payload
Variant	Dataset
Domains	–
Collision	–

Projector Light (`projector_light`)

projector_light


Type	light
Variant	Dataset
Domains	–
Collision	–

Raph Shovel (`raph_shovel`)


Type	payload
Variant	Dataset
Domains	–
Collision	–

Primitive Shapes

Random Shape (`random_shape`)

random_shape


Type	common
Variant	Primitive
Domains	–
Collision	–
Density	1000 kg/m³

Structures & Containers

Solar Panel (`solar_panel`)

solar_panel


Type	common
Variant	Dataset
Domains	–
Collision	sdf
Density	500 kg/m³

Scenery

Environmental scenery registered in the Space Robotics Bench. Includes terrains, facilities, and other scene elements.

Overview

Facilities

Iss Gateway Scenery (`iss_gateway_scenery`)


Type	extravehicular
Variant	Dataset
Domains	orbit
Sub-assets	iss, gateway

The ISSGatewayScenery entity can be switched to a single static station USD through station_template_usd_path while keeping static collision geometry for stable benchmark scenes. Segment handrails can be aligned with benchmark waypoint paths through handrail_segment_specs; handrail_specs remains available as a legacy shorthand for X-axis rails.

Kibou (`kibou`)

kibou kibou


Type	intravehicular
Variant	Dataset
Domains	orbit
Sub-assets	lamp

Zeroglab (`zeroglab`)

zeroglab


Type	intravehicular
Variant	Dataset
Domains	orbit
Sub-assets	lamp

Terrain

Ground Plane (`ground_plane`)

ground_plane


Type	terrain
Variant	Primitive
Domains	asteroid, earth, mars, moon

Lunalab (`lunalab`)

lunalab lunalab


Type	subterrane
Variant	Dataset
Domains	asteroid, earth, mars, moon
Sub-assets	terrain, lamp, boulder

Lunar Site Apollo15 (`lunar_site_apollo15`)


Type	terrain
Variant	–
Domains	moon

Lunar Site Apollo17 (`lunar_site_apollo17`)


Type	terrain
Variant	–
Domains	moon

Lunar Site Lunar Default (`lunar_site_lunar_default`)


Type	terrain
Variant	–
Domains	moon

Lunar Site Shackleton Rim (`lunar_site_shackleton_rim`)


Type	terrain
Variant	–
Domains	moon

Lunar Site Tycho Central Peak (`lunar_site_tycho_central_peak`)


Type	terrain
Variant	–
Domains	moon

Mars Surface (`mars_surface`)

mars_surface


Type	terrain
Variant	–
Domains	mars

Martian Site Gale (`martian_site_gale`)


Type	terrain
Variant	–
Domains	mars

Martian Site Jezero (`martian_site_jezero`)


Type	terrain
Variant	–
Domains	mars

Martian Site Mars Default (`martian_site_mars_default`)


Type	terrain
Variant	–
Domains	mars

Martian Site Olympus Mons Flank (`martian_site_olympus_mons_flank`)


Type	terrain
Variant	–
Domains	mars

Martian Site Oxia Planum (`martian_site_oxia_planum`)


Type	terrain
Variant	–
Domains	mars

Moon Surface (`moon_surface`)

moon_surface


Type	terrain
Variant	–
Domains	moon

Oberpfaffenhofen (`oberpfaffenhofen`)

oberpfaffenhofen


Type	terrain
Variant	Dataset
Domains	asteroid, earth, mars, moon

Plane (`plane`)

plane plane


Type	terrain
Variant	Primitive
Domains	asteroid, earth, mars, moon
Sub-assets	obstacle

Terrain Generator (`terrain_generator`)


Type	terrain
Variant	–
Domains	asteroid, earth, mars, moon

Integrations & Interfaces

The Space Robotics Bench has a modular design that supports integrations & interfaces with external frameworks and tools. This enables researchers and developers to leverage the high-fidelity environments while using their preferred toolchains for research, development, and validation.

At the moment, SRB natively supports the following categories of integrations/interfaces:

ROS 2 & Space ROS - Interface with the wide ecosystem of robotics tools and libraries
Reinforcement Learning (RL) - Train and evaluate policies using popular RL algorithms
Imitation Learning (IL) - Learn policies from collected demonstrations
Extended Reality (XR) - Immersive visualization and teleoperation through OpenXR headsets

Integration — Reinforcement Learning

The Space Robotics Bench is designed with Robot Learning research in mind, with a particular emphasis on Reinforcement Learning (RL). All environments follow the standard Gymnasium API, making them compatible with most modern RL frameworks. This architecture enables rapid prototyping, training, and evaluation of policies across diverse space robotics tasks, which makes SRB particularly suitable for comparing the generalization capabilities of novel RL algorithms.

Motivation

RL has emerged as a promising approach for developing autonomous behaviors in complex robotic systems, especially for space applications where manual control may be limited by communication delays or environmental uncertainties. SRB provides a comprehensive set of environments that are specifically designed to facilitate RL research in space robotics, offering several key features:

Diverse Tasks - Collection of various space robotics tasks that range from simple navigation to complex mobile manipulation
Parallel Training - All environments support parallel training across multiple instances for efficient data collection
Procedural Generation & Domain Randomization - Generalization capabilities of RL algorithms can be put to the test using procedurally generated environments that are randomized across multiple dimensions
Unified Benchmarking - RL algorithms can be compared across a set of reproducible tasks with consistent evaluation metrics

Available Integrations

We provide official integrations with the following RL frameworks:

Dreamer [JAX] - Model-based RL algorithm
Stable-Baselines3 (SB3) & SBX [PyTorch & JAX] - Popular implementation of RL algorithms
skrl [PyTorch & JAX] - Implementation of single- and multi-agent RL algorithms
TD-MPC2 [PyTorch] - Scalable model-based RL with strong sample efficiency
RSL-RL [PyTorch] - Fast on-policy algorithms (PPO) used widely in legged locomotion research

Unsupported Frameworks

ACME is intentionally not listed as a supported SRB integration. SRB framework adapters run in the same Python process as Isaac Sim, and the current SRB Docker runtime uses Python 3.12. The ACME runtime stack currently depends on packages that do not resolve for that runtime, including dm-reverb, dm-launchpad, and the old jaxlib==0.4.3 pin from dm-acme[jax].

How to Get Started?

If you want to start using the Space Robotics Bench for Reinforcement Learning, follow these steps:

Installation
Basic Usage
RL Workflow

Integration — Imitation Learning

The Space Robotics Bench supports Imitation Learning (IL) as a complementary paradigm to Reinforcement Learning. IL is particularly useful in space robotics where reward engineering is difficult, where sparse-reward tasks stall standard RL, or where expert trajectories (from teleoperation, planners, or prior policies) are available to bootstrap learning.

Motivation

Several characteristics of space robotics tasks make IL an attractive option:

Reward sparsity - Assembly, sample collection, and docking tasks often have reward signals that only appear at the very end of an episode, making RL credit assignment slow.
Safety-critical actions - Demonstrations encode a known-safe envelope of behaviors, reducing the risk of catastrophic exploration on expensive hardware.
Low-latency deployment - A policy distilled from demonstrations can be deployed to flight-grade hardware without the exploration noise that RL requires.
Hybrid workflows - IL policies can bootstrap RL indirectly — e.g. by rolling out an IL policy to collect additional demonstrations or by shaping curricula — even though RL checkpoints cannot be warm-started directly from IL checkpoints (the frameworks use different checkpoint formats).

Available Integrations

SRB integrates the following IL stack:

robomimic [PyTorch] - Reference implementation of BC (MLP/RNN/Transformer), BCQ, CQL, IQL, TD3-BC, GL, HBC, IRIS, and Diffusion Policy with a standard HDF5 dataset format.

All other frameworks already integrated for RL (Dreamer, TD-MPC2, Stable-Baselines3, SBX, skrl, RSL-RL) can also consume SRB-collected datasets for offline or demonstration-guided training, typically by using their replay buffer loading utilities.

Data Format

Demonstrations are stored in the robomimic-compatible HDF5 layout:

Per-episode groups containing obs/, next_obs/, actions, rewards, and dones datasets (plus a per-episode success attribute).
Observation keys match the SRB observation group structure (STATE, PROPRIO, COMMAND, STATE_PRIVILEGED, etc.).
Success flags are stored per-episode to support --success_only filtering at collection time.

Datasets written by srb agent collect are directly loadable by srb agent learn without any conversion step.

Integration — Extended Reality (XR)

The Space Robotics Bench supports Extended Reality (XR) through Isaac Sim’s OpenXR pipeline. This enables immersive teleoperation, demonstration collection, and scene inspection using commodity VR/AR headsets, which is especially valuable for designing human-in-the-loop workflows for space robotics.

Motivation

Immersive teleoperation - Operators can place themselves inside the simulated scene, giving them accurate depth perception and natural six-DoF control — properties that are difficult to match with a screen-and-mouse setup.
High-fidelity demonstrations - Demonstrations collected in XR produce smoother, more physically plausible trajectories, which translates into better downstream Imitation Learning performance.
Mission rehearsal - Operators can walk through a Martian, Lunar, or orbital scene before mission execution to validate assumptions about scale, lighting, and hardware placement.
Outreach & education - Space robotics is inherently visual; XR lets non-expert audiences experience SRB environments at full fidelity.

Hardware & Software Requirements

XR mode relies on Isaac Sim’s OpenXR runtime. Any OpenXR-compliant headset supported by Isaac Sim should work — this notably includes Meta Quest (via Link/Air Link), Valve Index, HTC Vive, and Windows Mixed Reality devices. Consult the Isaac Sim documentation for the current XR support matrix and runtime configuration.

Enabling XR

Two CLI flags control XR behavior:

--xr — enables the XR session for VR/AR applications.
--rendering_mode xr — selects the dedicated XR rendering preset (optimized for VR frame rates and stereo rendering).

These flags are available for every simulation-based agent subcommand (zero, rand, teleop, ros, train, eval, collect). They control rendering and headset display only — action input still comes from whichever --teleop_device is configured (keyboard, spacemouse, ros, gamepad, haptic).

Visualization of Trained Policies in XR

--xr can be combined with eval to observe a trained policy from inside the scene — a useful debugging aid when behavior is qualitatively odd:

srb agent eval --env peg_in_hole --algo sbx_ppo --xr --rendering_mode xr

Immersive Teleoperation

Pair --xr with a standard teleop device for an immersive viewpoint while controlling the robot through the existing input modality. The headset provides depth perception and natural head motion; the device drives the actions:

srb agent teleop --env peg_in_hole --xr --rendering_mode xr \
  --teleop_device spacemouse

Demonstration Collection in XR

The same flags apply to collect, producing demonstrations whose observation stream reflects a fully visualized scene:

srb agent collect --env peg_in_hole --xr --rendering_mode xr \
  --teleop_device spacemouse --num_episodes 50 --success_only

Tips

Prefer env.num_envs=1 in XR — parallel environments multiply the rendering cost and can drop below the frame rate needed for comfort.
If stereo rendering causes motion sickness, lower env.num_envs or switch back to --rendering_mode performance and use the headset only for inspection.
XR sessions record wall-clock time, so long teleoperation bouts produce correspondingly long datasets — use --horizon to cap episode length.
XR controller-as-teleop-input is not currently wired into SRB’s teleop device set; track upstream Isaac Sim XR additions if this is a future requirement.

How to Get Started?

If you want to start using the Space Robotics Bench with Extended Reality, follow these steps:

Installation
Basic Usage
Launch any simulation-based agent subcommand with the XR flags (see Enabling XR above)

Interface — ROS 2 & Space ROS

All environments and workflows in the Space Robotics Bench support the ROS 2 interface, which allows you to communicate with each environment instance using standard ROS 2 middleware interfaces. In doing so, you can collect sensor data, control robots, and interact with the simulation instances using the vast ecosystem of ROS tools and libraries.

Motivation

ROS 2 has become the de facto standard for developing robotic systems across various domains. The Space Robotics Bench provides a comprehensive ROS 2 interface that enables seamless integration between the simulation environment and the broader ROS ecosystem, offering several key benefits:

Compatibility with existing tools - Leverage the rich ecosystem of ROS tools like RViz2, tf2, and rosbag2 for visualization, debugging, and data recording
Workflow continuity - Develop algorithms and control systems that can transition smoothly from simulation to real hardware using the same interfaces
Distributed architecture - Take advantage of ROS’s node-based architecture to distribute computation and simulation across different processes or even machines
Community standardization - Utilize standard message types (geometry_msgs, sensor_msgs, …) that are widely understood and supported in the robotics community

Space ROS

Space ROS is an initiative aimed at extending the ROS ecosystem for applications beyond Earth. The ROS 2 interface of SRB is fully compatible with Space ROS, allowing you to develop and validate autonomous systems across a wide range of diverse extraterrestrial scenarios. In fact, the Space Robotics Bench is a spiritual continuation of the Parallel ProcGen Environments project developed during the NASA Space ROS Sim Summer Sprint Challenge.

You can integrate SRB with Space ROS in two ways:

Direct: Run the ROS 2 interface of SRB using the Space ROS middleware stack alongside the rest of your Space ROS nodes, which can be achieved simply by sourcing your Space ROS environment before running SRB
Indirect (recommended): Run the ROS 2 interface of SRB using the standard ROS 2 middleware stack and communicate with your Space ROS nodes using the standard ROS 2 interfaces (no additional setup required)

Implementation

The ROS 2 interface is implemented through a dynamically configured bridge node with the following key features:

Dynamic ROS interfaces - All publishers, subscribers, and services are created automatically based on the selected simulation environment without any manual configuration
Parallel environment support - Every parallel simulation instance can be managed through a separate ROS namespace for seamless parallelization, while global control is also available
Standard message translation - Environment states, sensory outputs, and control commands are automatically translated from simulation tensors to appropriate ROS interfaces
Complete access - All simulation entities (robots, sensors, objects, …) are automatically detected, and their interfaces are exposed to ROS alongside task-specific MDP rewards and signals

How to Get Started?

If you want to start using the Space Robotics Bench with ROS 2, follow these steps:

Installation
Basic Usage
ROS 2 Workflow

System Requirements

Hardware Requirements

The hardware requirements for Space Robotics Bench are inherited from the Isaac Sim requirements. With careful tuning, it is possible to run the included environments on lower-spec systems. However, the performance of some workflows might be limited. The bare minimum requirements are listed below:

Component	Requirement
CPU	x86_64
GPU	NVIDIA RTX
RAM	16 GB
VRAM	4 GB (12 GB for all workflows)
Disk Space	32 GB

This project requires a dedicated NVIDIA GPU with RT Cores (RTX series). Isaac Sim does not support GPUs from other vendors (AMD, Intel) or older NVIDIA GPUs without RT Cores.

Declared platform policy: linux/amd64+nvidia-rtx. This is a source support declaration, not runtime acceptance evidence. Linux arm64/aarch64 installer paths remain source-present but unverified and unsupported until their frozen install, container, Isaac/GPU, and end-to-end execution gates pass.

Software Requirements

A Linux-based OS with an appropriate NVIDIA driver is required to use Space Robotics Bench. Other operating systems might be functional, but they are not officially supported. Please let us know if you confirm functionality on other non-listed systems.

Component	Requirement
OS (Native Installation)	Ubuntu 22.04/24.04
OS (Docker Installation)	Linux (`X11` required only for GUI workflows)
NVIDIA Driver	>=580
Python	3.12.*
Isaac Sim	6.0 (built from source)
Isaac Lab	matched to Isaac Sim 6.0

Note: SRB requires Isaac Sim 6.0 built from source via the provided install script — a standard Omniverse-launcher installation is not compatible. The Installation guide handles this step for you.

NVIDIA Driver

Official instructions: Driver Installation Guide — Choose an Installation Method

Install the NVIDIA driver by following the official instructions above or through your distribution package manager.

… continue with Installation

Installation

Before proceeding, ensure your system meets the system requirements.

Just want to try SRB quickly? Use the Docker path with a locally built image or a release image selected by its complete sha256 digest.

Installation Methods

SRB supports three installation methods, each with different trade-offs:

A. Native

✅ Full system integration
✅ Smooth development experience
❗ Complex setup process
❗ Potential dependency conflicts

B. Docker (Recommended)

✅ Simple installation & deployment
✅ Reproducible environment & easy to update
⚠️ Moderate development experience (via Dev Containers)
✅ Includes helper scripts for GUI, headless, and ephemeral sessions
✅ Least-privilege profile for lightweight commands
❗ Isaac Sim’s trusted profile grants host-adjacent access (not suitable for HPC)

C. Apptainer/Singularity

✅ Deployable to HPC clusters
❗ Requires local Docker to build the Apptainer image (.sif)
❗ Limited interactive development support

Temporary Docker sessions

The runner uses --rm, so the container filesystem is discarded when the process exits. The safe profile has no persistent host mounts. It requires an already built local image, or an explicitly supplied registry image pinned by its full sha256 digest; it never downloads a mutable latest fallback.

Do not pipe a mutable branch’s setup script directly into a shell. Clone or otherwise authenticate the repository first, inspect the runner, and follow the Docker installation guide.

Installation — Native

This guide covers installing SRB natively on your system without containerization. Although this approach simplifies development, it requires more manual setup and decreases reproducibility.

1. Install Prerequisites

Install the required system packages:

sudo apt-get update
sudo apt-get install -y git git-lfs gcc-11 g++-11 cmake build-essential curl unzip xz-utils python3 jq

If you cannot use sudo, you can install git-lfs locally:

mkdir -p ~/.local/bin /tmp/git-lfs-install
curl -sSL https://github.com/git-lfs/git-lfs/releases/download/v3.6.1/git-lfs-linux-amd64-v3.6.1.tar.gz -o /tmp/git-lfs.tar.gz
tar xzf /tmp/git-lfs.tar.gz -C /tmp/git-lfs-install
cp /tmp/git-lfs-install/git-lfs-*/git-lfs ~/.local/bin/
rm -rf /tmp/git-lfs.tar.gz /tmp/git-lfs-install

Ensure ~/.local/bin is in your PATH.

2. Clone the Repository

git clone --recurse-submodules https://github.com/AndrejOrsula/space_robotics_bench.git
cd space_robotics_bench

If you prefer not to use git submodules, you can clone normally and then initialize asset repos from a Hugging Face mirror:

git clone https://github.com/AndrejOrsula/space_robotics_bench.git
cd space_robotics_bench
./assets/update.bash --source hf --repo-id <owner>/<repo>
./assets/update.bash --source hf --repo-id <owner>/<graphics-repo> --path docs/src/_graphics

Note: The --source hf path is for cloning from a Hugging Face mirror — substitute your own mirror’s repository IDs for <owner>/<repo>. If you do not have a mirror, prefer the default --recurse-submodules clone above, which needs no substitution.

All remaining commands in this guide assume your current working directory is the repository root.

3. Install NVIDIA Isaac Sim

Build Isaac Sim from source using the provided script:

./scripts/install_isaacsim.bash "$HOME/isaac-sim"

The build requires sudo access, significant time (~7 min), and disk space (~25 GB).

Environment Variables

Set the following environment variables in your shell configuration. Use the values printed by the install script — particularly VK_DRIVER_FILES, which is auto-detected for your system.

Also ensure ~/.local/bin is on your PATH, because the optional CLI setup step installs wrapper executables there by default.

`bash`

cat >> ~/.bashrc << 'EOF'
export PATH="$HOME/.local/bin:$PATH"
export ISAACSIM_PYTHON="$HOME/isaac-sim/python.sh"
export ISAACSIM_PATH="$HOME/isaac-sim"
export ISAAC_PATH="$HOME/isaac-sim"
export CARB_APP_PATH="$HOME/isaac-sim/kit"
export EXP_PATH="$HOME/isaac-sim/apps"
export LD_PRELOAD="$HOME/isaac-sim/kit/libcarb.so"
export VK_DRIVER_FILES="/usr/share/vulkan/icd.d/nvidia_icd.json"
export OMNI_SERVER="https://omniverse-content-staging.s3-us-west-2.amazonaws.com/Assets/Isaac/6.0"
export OMNI_KIT_ALLOW_ROOT="1"
EOF
source ~/.bashrc

`zsh`

cat >> ~/.zshrc << 'EOF'
export PATH="$HOME/.local/bin:$PATH"
export ISAACSIM_PYTHON="$HOME/isaac-sim/python.sh"
export ISAACSIM_PATH="$HOME/isaac-sim"
export ISAAC_PATH="$HOME/isaac-sim"
export CARB_APP_PATH="$HOME/isaac-sim/kit"
export EXP_PATH="$HOME/isaac-sim/apps"
export LD_PRELOAD="$HOME/isaac-sim/kit/libcarb.so"
export VK_DRIVER_FILES="/usr/share/vulkan/icd.d/nvidia_icd.json"
export OMNI_SERVER="https://omniverse-content-staging.s3-us-west-2.amazonaws.com/Assets/Isaac/6.0"
export OMNI_KIT_ALLOW_ROOT="1"
EOF
source ~/.zshrc

`fish`

fish_add_path "$HOME/.local/bin"
set -Ux ISAACSIM_PYTHON "$HOME/isaac-sim/python.sh"
set -Ux ISAACSIM_PATH "$HOME/isaac-sim"
set -Ux ISAAC_PATH "$HOME/isaac-sim"
set -Ux CARB_APP_PATH "$HOME/isaac-sim/kit"
set -Ux EXP_PATH "$HOME/isaac-sim/apps"
set -Ux LD_PRELOAD "$HOME/isaac-sim/kit/libcarb.so"
set -Ux VK_DRIVER_FILES "/usr/share/vulkan/icd.d/nvidia_icd.json"
set -Ux OMNI_SERVER "https://omniverse-content-staging.s3-us-west-2.amazonaws.com/Assets/Isaac/6.0"
set -Ux OMNI_KIT_ALLOW_ROOT "1"

VK_DRIVER_FILES depends on your system. Common locations:

/usr/share/vulkan/icd.d/nvidia_icd.json (Ubuntu with driver packages)
/etc/vulkan/icd.d/nvidia_icd.json (created by the script if neither exists)

If Isaac Sim fails with “Failed to create any GPU devices”, verify this path: ls -la "$VK_DRIVER_FILES".

4. Install NVIDIA Isaac Lab

./scripts/install_isaaclab.bash "$HOME/isaaclab"

The script also resolves numpy version conflicts and installs array_api_compat.

5. Install Blender 4.5 with SimForge

Official instructions: Blender — Install from blender.org

./scripts/install_blender.bash "$HOME/blender"

The script downloads Blender, creates a symlink at ~/.local/bin/blender, and installs SimForge into Blender’s bundled Python.

Avoid installing Blender through Snap, as it prevents integration of required Python dependencies.

6. Install RL Frameworks (Optional)

The following frameworks require source installation. Each is optional — install only the ones you need. Most scripts require ISAACSIM_PYTHON to be set, so source your shell configuration first if you haven’t already.

6a. DreamerV3

./scripts/install_dreamerv3.bash "$HOME/dreamerv3"

6b. TD-MPC2

TD-MPC2 is not pip-installed — only its source code is cloned. Python dependencies (tensordict, torchrl, etc.) are installed later via pyproject.toml.

./scripts/install_tdmpc2.bash "$HOME/tdmpc2"

The script pins upstream TD-MPC2 and applies the recorded backports from srb/integrations/tdmpc2/patches/ (currently upstream’s Q-ensemble weight-initialization fix, which otherwise leaves the entire Q ensemble on PyTorch’s default initialization). Upstream’s dependency-pin commit is deliberately not adopted, because it would downgrade the Torch/TensorDict/TorchRL matrix that Isaac Sim provides. SRB refuses to construct a TD-MPC2 agent on a checkout without the backport, so use this script (or reproduce the pin exactly as the error message describes) rather than a plain git clone.

Set the TDMPC2_PATH environment variable:

# Bash
echo "export TDMPC2_PATH='$HOME/tdmpc2/tdmpc2'" >> ~/.bashrc && source ~/.bashrc

# Zsh
echo "export TDMPC2_PATH='$HOME/tdmpc2/tdmpc2'" >> ~/.zshrc && source ~/.zshrc

# Fish
set -Ux TDMPC2_PATH "$HOME/tdmpc2/tdmpc2"

6c. robomimic

./scripts/install_robomimic.bash "$HOME/robomimic"

6d. rsl_rl

./scripts/install_rsl_rl.bash "$HOME/rsl_rl"

7. Install the Space Robotics Bench

"$ISAACSIM_PYTHON" -m pip install --editable ".[all]"

Note: The all extra installs optional dependencies for all workflows. See pyproject.toml to adjust.

Setup CLI

./scripts/setup_cli.bash

The setup script copies srb, simforge, and space_robotics_bench shims into ~/.local/bin by default and configures shell completions for detected shells.

Requirements for this step:

ISAACSIM_PYTHON must already be set and executable
the destination directory (default: ~/.local/bin) must already be on your PATH
python3 is used to discover the SRB cache directory for Hydra override completions
register-python-argcomplete is resolved from Isaac Sim first, then from your system PATH

# Only regenerate completions for specific shells
./scripts/setup_cli.bash --completions-only bash zsh fish

Hydra override completions depend on the SRB cache files. Generate them once after installation, then install the shell registrations:

srb completions generate
srb completions install
exec "$SHELL" -l

Note: jq is required for Hydra override completions. The srb CLI is always accessible via "$ISAACSIM_PYTHON" -m srb even if the setup script fails or you skip shim installation.

8. Verify Installation

If you encounter issues, refer to the Troubleshooting section below.

Isaac Sim

"$ISAACSIM_PYTHON" -c "
from isaacsim import SimulationApp
sim = SimulationApp({'headless': True})
print('Isaac Sim OK — is_running:', sim.is_running())
sim.close()
"

Note: The first launch takes longer due to shader compilation.

Isaac Lab

"$ISAACSIM_PYTHON" -m pip show isaaclab

Blender

blender --version

Space Robotics Bench

"$ISAACSIM_PYTHON" -m srb --help
command -v srb
srb --help
"$ISAACSIM_PYTHON" -m srb cache update

Run a Quick Simulation

"$ISAACSIM_PYTHON" -m srb agent zero --headless -e sample_collection --cfg ignore --perf --perf_duration 10

You should see a performance report with steps per second at the end.

… continue with Basic Usage

Troubleshooting

“Failed to create any GPU devices”

VK_DRIVER_FILES does not point to a valid Vulkan ICD file.

# Check current value
ls -la "$VK_DRIVER_FILES"

# Find the correct path
find /etc/vulkan /usr/share/vulkan -name "nvidia_icd.json" 2>/dev/null

Update VK_DRIVER_FILES in your shell configuration to the correct path.

“Application failed to start”

The DISPLAY environment variable is not set. For headless servers with a virtual display:

# Bash/Zsh
echo 'export DISPLAY=:99' >> ~/.bashrc  # or ~/.zshrc

# Fish
set -Ux DISPLAY :99

numpy version conflicts

The install_isaaclab.bash script resolves these automatically. If errors reappear after manual package changes:

"$ISAACSIM_PYTHON" -m pip install "numpy>=2.1" array_api_compat

Extras

Development

To improve your development experience, consider configuring your IDE.

Installation — Docker (Recommended)

Using SRB inside Docker is recommended for most users, as it provides an isolated, reproducible environment that is fully pre-configured.

1. Clone the Repository

First, clone the SRB repository with all submodules:

git clone --recurse-submodules https://github.com/AndrejOrsula/space_robotics_bench.git
cd space_robotics_bench

If you prefer not to use git submodules, you can clone normally and then initialize asset repos from a Hugging Face mirror:

git clone https://github.com/AndrejOrsula/space_robotics_bench.git
cd space_robotics_bench
./assets/update.bash --source hf --repo-id <owner>/<repo>
./assets/update.bash --source hf --repo-id <owner>/<graphics-repo> --path docs/src/_graphics

2. Install Docker Engine & NVIDIA Container Toolkit

Official instructions: Install Docker Engine

Official instructions: Linux post-installation steps for Docker Engine

Official instructions: Installing the NVIDIA Container Toolkit

For the standard interactive SRB workflow, your host should also have:

a working NVIDIA driver with nvidia-smi
xauth installed if you want Isaac Sim GUI forwarding (sudo apt-get install -y xauth)
an X11 session with DISPLAY set if you want GUI windows from the container

Install Docker Engine and NVIDIA Container Toolkit either by following the official instructions above or using the provided convenience script:

./.docker/host/install_docker.bash

If the installer adds your user to the docker group, start a new login session before relying on passwordless docker commands. Until then, the SRB helper scripts automatically fall back to sudo docker ... when needed.

3. Run

Build the repository’s local image, then run it with the provided script:

./.docker/build.bash
./.docker/run.bash

The normal runner resolves the local tag to its immutable image ID. It does not fall back to a remote tag. The default safe profile starts an interactive container with isolated networking and IPC, dropped capabilities, no host devices, and no host mounts. This profile is suitable for lightweight commands that do not need Isaac Sim host integration:

DOCKER_TTY=false ./.docker/run.bash srb --help

To use an already reviewed remote release, supply its complete registry digest; mutable tags are rejected:

SRB_DOCKER_IMAGE='<registry>/<image>@sha256:<64-hex-digest>' ./.docker/run.bash srb --help

Isaac Sim development needs host-adjacent access. Select that trust boundary explicitly:

SRB_DOCKER_PROFILE=trusted-simulator ./.docker/run.bash

The trusted simulator profile enables privileged mode, host networking and IPC, /dev, writable simulator caches, GUI forwarding, command history, and a writable /root/ws source mount by default. Use it only with a locally trusted image or an authenticated digest.

Useful variants:

# Headless trusted simulator session
SRB_DOCKER_PROFILE=trusted-simulator WITH_GUI=false ./.docker/run.bash

# CPU / non-GUI trusted development path
SRB_DOCKER_PROFILE=trusted-simulator WITH_GPU=false WITH_GUI=false ./.docker/run.bash

# Trusted simulator without writable source or shell history
SRB_DOCKER_PROFILE=trusted-simulator WITH_DEV_VOLUME=false WITH_HISTORY=false ./.docker/run.bash

# Force-enable NVIDIA runtime detection if host probing is incomplete
SRB_DOCKER_PROFILE=trusted-simulator WITH_GPU_FORCE_NVIDIA=true ./.docker/run.bash

# Non-interactive command execution, for CI/log capture/agent smoke tests
DOCKER_TTY=false ./.docker/run.bash bash -lc "srb --help"

To attach from another terminal while the container is still running:

./.docker/join.bash

Both run.bash and join.bash also accept a trailing command. Example:

SRB_DOCKER_PROFILE=trusted-simulator ./.docker/run.bash latest bash
./.docker/join.bash 0 fish

4. Verify Installation

Once you enter the Docker container, verify that everything works as expected. If you encounter any issues, please refer to the Troubleshooting guide.

Isaac Sim

Confirm that the container sees your GPU:

nvidia-smi

Confirm that you can launch Isaac Sim:

"$HOME/isaac-sim/isaac-sim.sh"

Note: The first launch might take a while because Isaac Sim needs to compile shaders and prepare the environment.

Space Robotics Bench

Verify that the srb command is available:

srb --help
srb cache update

GUI Forwarding

If you expect GUI access, also verify:

echo "$DISPLAY"
test -n "$XAUTHORITY" && ls -l "$XAUTHORITY"

… continue with Basic Usage

Extras

Build a New Docker Image

If you want to build a custom Docker image, you can use the provided script:

./.docker/build.bash

Join a Running Container

To join a running container from another terminal, use the provided script:

./.docker/join.bash

Development

The repository workspace is mounted by the explicit trusted simulator and development profiles, so edits made there persist on the host. The safe profile does not mount the checkout.

To improve your development experience, you can open the project as a Dev Container (guide).

For extra development-only environment variables or additional bind mounts, use ./.docker/dev.bash, which wraps ./.docker/run.bash and explicitly selects trusted-simulator. It prints a security warning before launch. The development wrapper defaults to LOG_LEVEL=info to keep Isaac and SimForge startup logs readable; use LOG_LEVEL=debug ./.docker/dev.bash ... when you need verbose diagnostics.

The development wrapper also persists Rust build caches across disposable containers. By default, host-side Cargo registry, git, and target caches live under ~/.cache/space_robotics_bench/docker, while container builds use CARGO_TARGET_DIR=/root/.cache/srb/cargo-target so prebuilt image artifacts in /root/.cargo/target remain visible. Override the host cache root with SRB_DOCKER_CACHE_DIR=/path/to/cache ./.docker/dev.bash ...; override the in-container target path with SRB_CARGO_TARGET_DIR=/path/in/container ./.docker/dev.bash ..., but do not set it to /root/.cargo/target.

Installation — Apptainer/Singularity

You can use SRB on HPC clusters or systems where Docker is not available by leveraging Apptainer. Apptainer is a containerization tool similar to Docker but designed for environments with stricter security policies or without Docker support.

Note: Building the Apptainer image (Step 2 below) still requires Docker on your local machine, since the .sif is converted from the SRB Docker image. Only the resulting .sif runs on the Docker-free target cluster — so build locally, then transfer.

Setup (Local)

1. Clone the Repository

First, clone the SRB repository with all submodules:

git clone --recurse-submodules https://github.com/AndrejOrsula/space_robotics_bench.git
cd space_robotics_bench

If you prefer not to use git submodules, you can clone normally and then initialize asset repos from a Hugging Face mirror:

git clone https://github.com/AndrejOrsula/space_robotics_bench.git
cd space_robotics_bench
./assets/update.bash --source hf --repo-id <owner>/<repo>
./assets/update.bash --source hf --repo-id <owner>/<graphics-repo> --path docs/src/_graphics

All remaining local-build commands in this guide assume your current working directory is the repository root.

2. Install Docker Engine & NVIDIA Container Toolkit

Official instructions: Install Docker Engine

Official instructions: Linux post-installation steps for Docker Engine

Official instructions: Installing the NVIDIA Container Toolkit

Install Docker Engine and NVIDIA Container Toolkit either by following the official instructions above or using the provided convenience script:

./.docker/host/install_docker.bash

This local build path still depends on Docker because the Apptainer image is produced from the repository Docker image first.

3. Install Apptainer

Official instructions: Installing Apptainer

Install Apptainer by following the official instructions above. For instance, you can use these commands on Ubuntu:

sudo add-apt-repository -y ppa:apptainer/ppa
sudo apt update
sudo apt install -y apptainer

4. Build the Apptainer Image

Now you can build the Apptainer image with the provided script:

./.docker/hpc/build.bash

The build script first runs ./.docker/build.bash, then converts the resulting Docker image into an Apptainer image at:

.docker/hpc/images/space_robotics_bench.sif

Deployment (HPC Cluster)

1. Transfer the Repository (with the Apptainer Image)

Transfer the local SRB repository to the HPC cluster using your preferred method (e.g., scp or rsync):

A. `scp`

scp -r /path/to/space_robotics_bench user@hpc-cluster:/path/to/space_robotics_bench

B. `rsync`

rsync -avr /path/to/space_robotics_bench user@hpc-cluster:/path/to/space_robotics_bench

2. SSH into the HPC Cluster

SSH into the HPC cluster:

ssh user@hpc-cluster

3. Run the Apptainer Image

Now you can run the Apptainer image with the provided script inside an interactive session (you might need make some adjustments based on your HPC environment):

# HPC interactive session
/path/to/space_robotics_bench/.docker/hpc/run.bash bash

Notes about run.bash:

it expects to run on a node where the module command is available
it loads tools/Singularity automatically
it mounts the repository at /root/ws
it persists Isaac Sim and SimForge caches under ${SCRATCH:-$HOME}/volumes/space_robotics_bench/
WITH_GPU=true by default and adds --nv; set WITH_GPU=false for CPU-only runs
override IMAGE_PATH if your .sif lives outside .docker/hpc/images/
append any command after run.bash; bash is just the common interactive example

4. Verify Installation

Once you enter the Apptainer container, verify that everything works as expected. If you encounter any issues, please refer to the Troubleshooting guide.

Isaac Sim

Confirm that the container sees your GPU:

# Inside Apptainer container
nvidia-smi

Confirm that you can launch Isaac Sim:

# Inside Apptainer container
"$HOME/isaac-sim/isaac-sim.sh"

Note: The first launch might take a while because Isaac Sim needs to compile shaders and prepare the environment.

Space Robotics Bench

Verify that the srb command is available:

# Inside Apptainer container
srb --help
srb cache update

… continue with Basic Usage

Extras

Schedule a SLURM Job

For long-running tasks or automated workflows, you can schedule a SLURM job that will automatically run the Apptainer image. It is highly recommended that you adjust the script to your needs before submitting the job:

# HPC login node
/path/to/space_robotics_bench/.docker/hpc/submit.bash [CMD]

By default, submit.bash writes the generated job script under $HOME/jobs/, then submits it with sbatch. Its baked-in resource request is only a template; review the partition, constraint, memory, wall time, and GPU settings before using it on your cluster.

Basic Usage

After successful installation, you are now ready to explore the Space Robotics Bench. This guide covers the essentials for getting started with the framework.

Native Installation — If the srb command is not available, you can use this syntax:

"$ISAACSIM_PYTHON" -m srb

1. List Registered Assets & Environments

Reference: srb ls — List Assets and Environments

As a first step, it is recommended that you list all registered assets, action groups, and tasks to get an overview of what SRB has to offer:

srb ls

After a while, you should see 4 tables printed in the terminal:

1. Assets: Simulation assets categorized under sceneries, objects, and robots (click to expand)

Sceneries - Terrains, space stations, …
Objects - Interactive objects, tools, …
Robots - Manipulators, mobile robots, …

┏━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Name         ┃ Type    ┃ Subtype            ┃ Parent Class      ┃ Asset Config    ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ mars_surface │ scenery │ terrain            │ Terrain           │ AssetBaseCfg    │
│ sample_tube  │ object  │ common             │ Object            │ RigidObjectCfg  │
│ scoop        │ object  │ tool               │ Tool              │ RigidObjectCfg  │
│ shadow_hand  │ object  │ tool               │ ActiveTool        │ ArticulationCfg │
│ ur10         │ robot   │ manipulator        │ SerialManipulator │ ArticulationCfg │
│ cubesat      │ robot   │ mobile_robot       │ OrbitalRobot      │ RigidObjectCfg  │
│ ingenuity    │ robot   │ mobile_robot       │ Multicopter       │ ArticulationCfg │
│ perseverance │ robot   │ mobile_robot       │ WheeledRobot      │ ArticulationCfg │
│ unitree_g1   │ robot   │ mobile_manipulator │ Humanoid          │ ArticulationCfg │
│ ...          │ ...     │ ...                │ ...               │ ...             │
└──────────────┴─────────┴────────────────────┴───────────────────┴─────────────────┘

Scenery asset (automatically registered as “mars_surface” scenery/terrain)

class MarsSurface(Terrain):
    ## Scenario - The asset is suitable for the Mars domain
    DOMAINS: ClassVar[Sequence[Domain]] = (Domain.MARS,)

    ## Model - Static asset
    asset_cfg: AssetBaseCfg = AssetBaseCfg(
        prim_path="{ENV_REGEX_NS}/mars_surface",
        ## Spawner procedurally generates SimForge models
        spawn=SimforgeAssetCfg(
            assets=[simforge_foundry.MarsSurface()],
            collision_props=CollisionPropertiesCfg(),
        ),
    )

Object asset (automatically registered as “sample_tube” object/common)

class SampleTube(Object):
    ## Model - Rigid object affected by physics
    asset_cfg: RigidObjectCfg = RigidObjectCfg(
        prim_path="{ENV_REGEX_NS}/sample",
        ## Spawner loads a static USD file
        spawn=UsdFileCfg(
            usd_path=(
                SRB_ASSETS_DIR_SRB_OBJECT.joinpath("sample_tube.usdc").as_posix()
            ),
            collision_props=CollisionPropertiesCfg(),
            mesh_collision_props=MeshCollisionPropertiesCfg(
                mesh_approximation="convexDecomposition"
            ),
            rigid_props=RigidBodyPropertiesCfg(),
            mass_props=MassPropertiesCfg(density=1500.0),
        ),
    )

Robot asset (automatically registered as “franka” robot/manipulator)

class Franka(SerialManipulator):
    ## Model - Articulation with several links connected by joints
    asset_cfg: ArticulationCfg = ArticulationCfg(
        prim_path="{ENV_REGEX_NS}/franka",
        ## Spawner loads a static USD file
        spawn=UsdFileCfg(
            usd_path=SRB_ASSETS_DIR_SRB_ROBOT.joinpath("manipulator")
            .joinpath("franka_arm.usdz")
            .as_posix(),
            ...
        ),
        ## Initial joint configuration of the robot
        init_state=ArticulationCfg.InitialStateCfg(
            joint_pos={
                "panda_joint1": 0.0,
                "panda_joint2": 0.0,
                "panda_joint3": 0.0,
                "panda_joint4": radians(-90.0),
                "panda_joint5": 0.0,
                "panda_joint6": radians(90.0),
                "panda_joint7": radians(45.0),
            },
        ),
        ...
    )
    ## End effector - The default hand is separate to allow for easy replacement
    end_effector: Tool | None = FrankaHand()

    ## Action modes — coded-in alternatives selectable via
    ## `env.robot.action_mode=<key>`. The first entry is the default.
    action_modes: ClassVar[dict[str, ActionGroup]] = {
        "task_space_delta": InverseKinematicsActionGroup(
            DifferentialInverseKinematicsActionCfg(
                asset_name="robot",
                joint_names=["panda_joint[1-7]"],
                base_name="panda_link0",
                body_name="panda_link7",
                controller=DifferentialIKControllerCfg(
                    command_type="pose",
                    use_relative_mode=True,
                    ik_method="dls",
                ),
                scale=0.1,
                body_offset=DifferentialInverseKinematicsActionCfg.OffsetCfg(),
            ),
        ),
        "operational_space_pose": OperationalSpaceControlActionGroup(
            OperationalSpaceControllerActionCfg(
                asset_name="robot",
                joint_names=["panda_joint[1-7]"],
                body_name="panda_link7",
                controller_cfg=OperationalSpaceControllerCfg(
                    target_types=["pose_rel"],
                    impedance_mode="fixed",
                    motion_stiffness_task=100.0,
                    motion_damping_ratio_task=1.0,
                    nullspace_control="position",
                    inertial_dynamics_decoupling=True,
                ),
                nullspace_joint_pos_target="center",
                position_scale=0.1,
                orientation_scale=0.1,
                body_offset=OperationalSpaceControllerActionCfg.OffsetCfg(),
            )
        ),
        "operational_space_hybrid": OperationalSpaceControlActionGroup(
            OperationalSpaceControllerActionCfg(
                asset_name="robot",
                joint_names=["panda_joint[1-7]"],
                body_name="panda_link7",
                controller_cfg=OperationalSpaceControllerCfg(
                    target_types=["pose_rel"],
                    impedance_mode="variable_kp",
                    motion_stiffness_limits_task=(10.0, 250.0),
                    motion_damping_ratio_task=1.0,
                    nullspace_control="position",
                    inertial_dynamics_decoupling=True,
                ),
                nullspace_joint_pos_target="center",
                position_scale=0.1,
                orientation_scale=0.1,
                stiffness_scale=120.0,
                body_offset=OperationalSpaceControllerActionCfg.OffsetCfg(),
            )
        ),
        "operational_space_impedance": OperationalSpaceControlActionGroup(
            OperationalSpaceControllerActionCfg(
                asset_name="robot",
                joint_names=["panda_joint[1-7]"],
                body_name="panda_link7",
                controller_cfg=OperationalSpaceControllerCfg(
                    target_types=["pose_rel"],
                    impedance_mode="variable",
                    motion_stiffness_limits_task=(10.0, 250.0),
                    motion_damping_ratio_limits_task=(0.5, 2.5),
                    nullspace_control="position",
                    inertial_dynamics_decoupling=True,
                ),
                nullspace_joint_pos_target="center",
                position_scale=0.1,
                orientation_scale=0.1,
                stiffness_scale=120.0,
                damping_ratio_scale=1.0,
                body_offset=OperationalSpaceControllerActionCfg.OffsetCfg(),
            )
        ),
        "joint_position": JointPositionRelativeActionGroup(),
    }
    ...
    ## Frames - Relevant frames for attaching the robot and mounting tool/sensors
    frame_base: Frame = Frame(prim_relpath="panda_link0")
    frame_flange: Frame = Frame(
        prim_relpath="panda_link7",
        offset=Transform(
            pos=(0.0, 0.0, 0.107),
            rot=rpy_to_quat(0.0, 0.0, -45.0),
        ),
    )

2. Action groups: Pre-configured action modes for robots and active tools

Actions for robots - Each robot (mobile or manipulator) exposes selectable action modes
Actions for active tools - Each active tool (e.g. gripper) exposes selectable action modes

┏━━━━━━━━━━━━━━━━━━━━┓
┃ Name               ┃
┡━━━━━━━━━━━━━━━━━━━━┩
│ body_acceleration  │
│ joint_position     │
│ joint_velocity     │
│ joint_effort       │
│ inverse_kinematics │
│ ...                │
└────────────────────┘

3. Hardware interfaces: Sim-to-real interface classes for physical robots

Hardware interfaces - Bridges that map agent actions and observations to real hardware (e.g. via ROS 2)

┏━━━━━━━━━━━━━━━━━━━━┓
┃ Name               ┃
┡━━━━━━━━━━━━━━━━━━━━┩
│ ros_cmd_vel        │
│ ros_imu            │
│ moveit_servo       │
│ moveit_gripper     │
│ ...                │
└────────────────────┘

4. Environments: Gymnasium environments for templates and tasks

Templates - Barebones environments that can be used as a starting point
Tasks - Goal-oriented environments that provide a specific scenario

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ID                               ┃ Entrypoint                   ┃ Config                             ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ _manipulation <template>         │ Task(ManipulationEnv)        │ TaskCfg(ManipulationEnvCfg)        │
│ sample_collection                │ Task(ManipulationEnv)        │ TaskCfg(ManipulationEnvCfg)        │
│ _aerial <template>               │ Task(AerialEnv)              │ TaskCfg(AerialEnvCfg)              │
│ _ground <template>               │ Task(GroundEnv)              │ TaskCfg(GroundEnvCfg)              │
│ _orbital <template>              │ Task(OrbitalEnv)             │ TaskCfg(OrbitalEnvCfg)             │
│ velocity_tracking                │ Task(GroundEnv)              │ TaskCfg(GroundEnvCfg)              │
│ locomotion_velocity_tracking     │ LocomotionTask(Task)         │ LocomotionTaskCfg(TaskCfg)         │
│ _aerial_manipulation <template>  │ Task(AerialManipulationEnv)  │ TaskCfg(AerialManipulationEnvCfg)  │
│ _ground_manipulation <template>  │ Task(GroundManipulationEnv)  │ TaskCfg(GroundManipulationEnvCfg)  │
│ _orbital_manipulation <template> │ Task(OrbitalManipulationEnv) │ TaskCfg(OrbitalManipulationEnvCfg) │
│ ...                              │ ...                          │ ...                                │
└──────────────────────────────────┴──────────────────────────────┴────────────────────────────────────┘

2. Teleoperate your 1^st Robot across Diverse Domains

Reference: srb agent teleop — Teleoperate Agent

Let’s start with the sample_collection environment, inspired by the Mars Sample Return mission, where you can manually control the Franka manipulator through your keyboard to collect sample tubes:

srb agent teleop --env sample_collection

Eventually, Isaac Sim will open with the selected environment, and you will be greeted in your terminal with a schematic of the teleoperation interface.

Teleoperation Interface (click to expand)

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃  Keyboard Scheme (focus the Isaac Sim window)  ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Reset: [ L ]                                   │
│ Decrease Gain [ O ]   │ Increase Gain: [ P ]   │
│ Event: [ R / K ]                               │
├────────────────────────────────────────────────┤
│ Translation                                    │
│             [ W ] (+X)            [ Q ] (+Z)   │
│               ↑                     ↑          │
│               │                     │          │
│  (-Y) [ A ] ←─┼─→ [ D ] (+Y)        ┼          │
│               │                     │          │
│               ↓                     ↓          │
│             [ S ] (-X)            [ E ] (-Z)   │
├────────────────────────────────────────────────┤
│ Rotation                                       │
│       [ Z ] ←————————(±X)————————→ [ X ]       │
│                                                │
│       [ T ] ↻————————(±Y)————————↺ [ G ]       │
│                                                │
│       [ C ] ↺————————(±Z)————————↻ [ V ]       │
└────────────────────────────────────────────────┘

Note: Most tasks employ action spaces that support direct teleoperation (e.g. via Inverse Kinematics). However, some tasks such as locomotion_velocity_tracking rely on low-level control of individual joints. In this case, direct teleoperation is not supported, and you will need to provide a control policy that maps your teleoperation commands to low-level control signals. Further instructions are provided in the section for Teleoperation via Policy.

Move to a Different Domain

Reference: Environment Configuration
Reference: Environment Configuration — Domain

What if we want to collect natural samples on the Moon instead? Luckily, you can easily configure many aspects of the environment through Hydra. For instance, you can adjust the domain and configure the sample asset to spawn procedurally generated Lunar rock samples (moon_rock):

srb agent teleop --env sample_collection env.domain=moon env.sample=moon_rock

3. Observe Random Agents in Action

Reference: srb agent rand — Random Agent

Now, let’s observe an environment where agents act based on random actions sampled uniformly from their action space, which is particularly useful for verifying that environments function as intended. To demonstrate a random agent, we will use the locomotion_velocity_tracking task that uses the Spot quadruped by default:

srb agent rand --env locomotion_velocity_tracking

Hint: Use --hide_ui option to disable most of the Isaac Sim UI, as shown in the video above.

Change the Robot

Reference: Environment Configuration — Robot

Selecting a different robot for any environment is as simple as adjusting the env.robot parameter. This particular environment supports all legged robots, so let’s try the Unitree G1 humanoid:

srb agent rand --env locomotion_velocity_tracking env.robot=unitree_g1

Simulate Multiple Robots

Reference: Environment Configuration — Parallelism

Many workflows benefit from running multiple parallel simulation instances. This can be achieved with the env.num_envs parameter. For instance, let’s try 16 instances of the Cassie biped:

srb agent rand -e locomotion_velocity_tracking env.robot=cassie env.num_envs=16

Hint: As you can see in the video above, all 16 robots share the same terrain. This is the default behaviour of locomotion_velocity_tracking (the default behaviour is task-specific). However, you can easily create a unique terrain for each robot by setting env.stack=false. This will automatically trigger the generation of 16 unique assets with different geometry and materials. Here, we speed up the procedural generation by disabling texture baking with SF_BAKER=0:
SF_BAKER=0 srb agent rand -e locomotion_velocity_tracking env.stack=false env.num_envs=16

4. Explore & Experiment with Environment Templates

Reference: Assets
Reference: Environment Configuration — Robot

Both sample_collection and locomotion_velocity_tracking are examples of tasks that implement specific goal-oriented scenarios. However, SRB also provides a set of environment templates that can serve as a foundation for exploring and experimenting with custom scenarios.

In general, each robot category has its own template:

Template	Description
`_manipulation`	Fixed-base manipulation with robotic arms
`_ground`	Ground traversal on planetary surfaces
`_aerial`	Aerial navigation above planetary surfaces
`_orbital`	Spaceflight maneuvers
`_ground_manipulation`	Mobile manipulation with ground-based robots
`_aerial_manipulation`	Mobile manipulation with flying robots
`_orbital_manipulation`	Mobile manipulation with spacecraft

With this in mind, let’s explore the _ground_manipulation template that combines the mobile Spot quadruped with Franka manipulator into an integrated mobile manipulation system:

srb agent rand -e _ground_manipulation

Diversify Robot Configurations

Robot configurations can also be randomized across multiple parallel instances. For example, you can combine a random Unitree quadruped (random_unitree_quadruped) with a random Universal Robots manipulator (random_ur_manipulator), ranging all the way from UR3 to UR30! For mobile manipulators, changing the mobile base and manipulator is separated into two parameters for more flexibility, namely env.robot.mobile_base and env.robot.manipulator:

srb agent rand -e _ground_manipulation env.robot.mobile_base=random_unitree_quadruped env.robot.manipulator=random_ur_manipulator env.num_envs=6 env.stack=true

Customize Payloads & End Effectors

Reference: srb agent zero — Zero Agent

Modifying only the robot might not be enough for your envisioned scenario. You might also want to customize the payload of mobile robots or the end effector of manipulators. Similar to previous examples, this can also be configured via env.robot for mobile robots and manipulators; or env.robot.mobile_base and env.robot.manipulator for mobile manipulators. The configuration is context-aware, and you can specify payloads and end effectors by separating them with a + sign, i.e. mobile_base+payload or manipulator+end_effector. For example, let’s combine Unitree Z1 manipulator with Shadow Hand end effector on top of Anymal D quadruped with the Cargo Bay payload:

srb agent zero -e _ground_manipulation env.robot.mobile_base=anymal_d+cargo_bay env.robot.manipulator=unitree_z1+shadow_hand

Hint: Both sides of the + must be specified — e.g. anymal_d+cargo_bay or unitree_z1+shadow_hand. The shorthand forms +payload/+end_effector (omitting the base) and mobile_robot+/manipulator+ (omitting the attachment) are not supported and raise an error.

And while the results might look ridiculous, the same level of customization is available across the board for all domains. Furthermore, aspects such as the pose of sensors and dimensionality of action spaces are adjusted automatically.

srb agent zero -e _aerial_manipulation env.robot.mobile_base=ingenuity env.robot.manipulator=franka+scoop

srb agent zero -e _orbital_manipulation env.robot.mobile_base=gateway env.robot.manipulator=canadarm3

5. Enable Visual Sensors

To maintain the performance of simulation, all visual sensors are disabled by default. Cameras are activated by adding visual to the env.active_sensors set. Each environment exposes a number of pre-configured cameras that provide both RGB and depth images (available via observations and middleware communication).

Let’s see a camera view of the Ingenuity helicopter on Mars:

srb agent teleop -e _aerial 'env.active_sensors=[proprioception,contact,transform,visual]'

Hint: You can also tweak env.camera_resolution (default (64, 64)), env.camera_data_types (default ("rgb", "depth")), and env.camera_update_period to control the camera streams.

What’s Next?

Everything you learned so far is just the tip of the iceberg, but it is applicable to all environments and workflows within the Space Robotics Bench. Yet, diving deeper into the codebase will allow you to customize and extend the environments further to suit your specific needs.

Depending on your interests, you are welcome to explore one or more of the following guides:

Workflows: ROS 2
Workflows: Reinforcement Learning
Contributing — New Assets
Contributing — New Tasks

Workflows

The Space Robotics Bench provides a variety of workflows that streamline the interaction with the simulation environments to accomplish specific goals like algorithm development, validation, or synthetic data generation.

Available Workflows

ROS 2 - Interface with the wide ecosystem of robotics tools and libraries
Reinforcement Learning - Train and evaluate policies using popular RL algorithms
Imitation Learning - Collect demonstrations and train policies offline
Sim-to-Real Transfer - Deploy policies trained in simulation onto real hardware
Terrain Landscaping Training Readiness - What the three landscaping tasks prove, and the gates that are still blocked

ROS 2 Workflow

The ROS 2 interface of SRB allows you to interact with the simulation environment using standard ROS 2 middleware interfaces. This enables you to develop and validate autonomous systems across a wide range of diverse scenarios through seamless integration with the rest of your ROS 2 or Space ROS nodes.

There are two options for using the ROS 2 interface of SRB:

Active ROS 2 Agent: A specific srb agent ros workflow that enables full control over the simulation through the ROS 2 middleware, including actions of robots (control commands)
Passive ROS 2 Interface: Any workflow can be configured to use the ROS 2 interface for publishing all available data and interacting with the simulation through services, but the actions are controlled via the selected workflow (teleoperation, random agent, RL training and evaluation, …)

1. Active ROS 2 Agent

This workflow is specifically designed for full control over the simulation, including the actions of robots (control commands). It is available via the srb agent ros command, and works with all SRB environments. Let’s try it with the peg_in_hole task and the visual sensors enabled:

srb agent ros -e peg_in_hole env.num_envs=2 'env.active_sensors=[proprioception,contact,transform,visual]'

For closed-loop controllers that compute one action from each received simulator state, run the ROS agent in synchronized mode:

srb agent ros --ros_sync -e peg_in_hole env.num_envs=2 'env.active_sensors=[proprioception,contact,transform,visual]'

Without --ros_sync, the active ROS agent samples and holds the latest received action each simulation step, while pacing the loop to the environment agent step period (env.agent_rate). This is useful for one-shot commands and teleoperation-like publishers. If the external controller publishes slower than the environment agent period, the simulator can still reuse cached actions for multiple steps. Use --ros_unthrottled only when you intentionally want the old fastest-possible sample-and-hold loop.

Action subscription topics use latest-only best-effort QoS (KEEP_LAST, depth 1). This keeps high-rate controller publishers from building a reliable backlog of stale commands.

Some tasks intentionally configure action or observation delays for policy robustness. For controller debugging, disable those explicitly, for example:

srb agent ros --ros_sync -e waypoint_navigation env.action_delay_steps=0 env.observation_delay_steps=0

List Topics

Once the simulation is running, you can open another terminal and list all available ROS 2 topics:

ros2 topic list

Hint: For Docker installation, you can use .docker/join.bash to enter the running container if you don’t have a local ROS 2 setup available.

You will see the following list of topics: (click to expand)

/clock
/parameter_events
/rosout
/srb/env0/action/cmd_vel
/srb/env0/action/event
/srb/env0/cam_base/camera_info
/srb/env0/cam_base/image_depth
/srb/env0/cam_base/image_rgb
/srb/env0/cam_base/pointcloud
/srb/env0/cam_scene/camera_info
/srb/env0/cam_scene/image_depth
/srb/env0/cam_scene/image_rgb
/srb/env0/cam_scene/pointcloud
/srb/env0/cam_wrist/camera_info
/srb/env0/cam_wrist/image_depth
/srb/env0/cam_wrist/image_rgb
/srb/env0/cam_wrist/pointcloud
/srb/env0/end_effector/joint_pos
/srb/env0/end_effector/joint_states
/srb/env0/reward
/srb/env0/reward/penalty_action_rate
/srb/env0/reward/penalty_undesired_robot_contacts
/srb/env0/reward/reward_align_peg_to_hole_primary
/srb/env0/reward/reward_align_peg_to_hole_secondary
/srb/env0/reward/reward_distance_end_effector_to_obj
/srb/env0/reward/reward_distance_peg_to_hole_bottom
/srb/env0/reward/reward_distance_peg_to_hole_entrance
/srb/env0/reward/reward_grasp
/srb/env0/reward/reward_lift
/srb/env0/robot/delta_twist
/srb/env0/robot/joint_states
/srb/env0/terminated
/srb/env0/truncated
/srb/env1/...          (mirrors env0)
/srb/envs/action/cmd_vel
/srb/envs/action/event
/srb/envs/end_effector/joint_pos
/srb/envs/robot/delta_twist
/tf
/tf_static

Most topics fall under two namespaces:

/srb/env[0-N]/...: Environment-specific topics that are unique to each environment instance
/srb/envs/...: Global topics that apply to all environment instances

Note: With a single environment (env.num_envs=1), the env{i}/ and envs/ prefixes are omitted and topics are published directly under /srb/... (e.g. /srb/reward, /srb/action/cmd_vel).

Let’s break down the most important topics:

Topic	Message Type	Description
`/clock`	`rosgraph_msgs/Clock`	Simulation time
`/tf`, `/tf_static`	`tf2_msgs/TFMessage`	Transformations for all scene entities
Control
`**/{robot_name}/{action_name}`	Robot-specific	Action-specific control
`**/action/cmd_vel`	`geometry_msgs/Twist`	Universal command (task-specific mapping)
`**/action/event`	`std_msgs/Bool`	Universal event (task-specific mapping)
Perception
`**/[{sensor_name}]/{data}`	Sensor-specific	Sensor data of all scene sensors
`**/{robot_name}/joint_states`	`sensor_msgs/JointState`	Joint states of all scene articulations
Environment (MDP)
`**/reward`	`std_msgs/Float32`	Total reward
`**/reward/{term_name}`	`std_msgs/Float32`	Reward components
`**/terminated`	`std_msgs/Bool`	Episode termination flag
`**/truncated`	`std_msgs/Bool`	Episode truncation flag

You can subscribe to topics to receive updates from the simulation. For example, you can echo the reward signal of the first environment instance:

ros2 topic echo /srb/env0/reward

data: 2.36441707611084
---
data: 2.3652756214141846
---
data: 2.36613130569458
---
data: 2.3669850826263428
---

Publish to Topics

You can also send control commands to the simulation by publishing to topics. For example, you can move robots in all environment instances by publishing a twist command:

ros2 topic pub --once /srb/envs/action/cmd_vel geometry_msgs/Twist "{linear: {z: -0.2}, angular: {z: 0.785}}"

2. Passive ROS 2 Interface

Alternatively, you can use the ROS 2 interface with any agent type by specifying the --interface ros flag. This allows you to collect sensor data and interact with the simulation instances while controlling the agent through the selected workflow. Let’s try it with the rand agent in the multi_sample_collection environment with 8 parallel instances and visual sensors enabled:

srb agent rand -e multi_sample_collection env.num_envs=8 'env.active_sensors=[proprioception,contact,transform,visual]' --interface ros

Note: The --interface X option accepts one or more interfaces, but Hydra key=value overrides placed directly after it are automatically detected and forwarded to Hydra.

Call Services

Upon listing the available services, you will see that each environment instance has a /srb/env{i}/reset service, and a global /srb/envs/reset service.

ros2 service list

You can call these services to reset a specific environment instance or all instances at once:

ros2 service call /srb/env0/reset std_srvs/srv/Empty

ros2 service call /srb/envs/reset std_srvs/srv/Empty

Reinforcement Learning Workflow

Reinforcement Learning (RL) is one of the primary focus areas of the Space Robotics Bench. While there are several RL frameworks with their unique peculiarities, SRB offers a unified interface for training and evaluating policies across a diverse set of space robotics tasks.

1. Train your 1^st RL Agent

Reference: srb agent train — Train Agent

The fastest way to get started with training an RL agent is by using the srb agent train command, which provides a streamlined interface for all integrated RL frameworks. In general, you want to specify the RL algorithm to use, the environment to train on, and the number of parallel environment instances used for rollout collection.

Let’s start with a simple landing environment using the sbx_ppo algorithm (PPO implementation of SBX). For now, omit the --headless flag so that you can observe the convergence in real time:

srb agent train --algo sbx_ppo --env landing env.num_envs=512 --hide_ui

As you begin to observe the training process, you can also monitor the progress in your terminal. After about 25M timesteps, you will see that the agent found a stable policy that successfully solves the task. Checkpoints are saved regularly, so you are free to stop the training process at any point by sending an interrupt signal (Ctrl+C in most terminals).

2. Evaluate your Agent

Reference: srb agent eval — Evaluate Agent

Once training is complete, you can evaluate your agent with the srb agent eval command:

srb agent eval --algo sbx_ppo --env landing env.num_envs=16

By default, the latest checkpoint from the training run is loaded for evaluation. However, you might want to run the evaluation for a checkpoint specified via --model:

srb agent eval --algo sbx_ppo --env landing env.num_envs=16 --model space_robotics_bench/logs/landing/sbx_ppo/${TIMESTAMP}/ckpt/${CHECKPOINT}

Note: Each training run writes to its own timestamped directory (logs/<env>/<algo>/<timestamp>/), and SB3/SBX checkpoints are saved under its ckpt/ subdirectory (.zip files).

3. Try a Different Algorithm

SRB directly supports several popular RL algorithms from different frameworks:

Algorithm Type	DreamerV3	TD-MPC2	Stable-Baselines3	SBX	RSL-RL	skrl
Model-based	dreamer	tdmpc2

On-Policy			sb3_a2c		rsl_rl_ppo	skrl_a2c
			sb3_ppo	sbx_ppo		skrl_ppo
			sb3_ppo_lstm			skrl_ppo_rnn
						skrl_rpo
			sb3_trpo			skrl_trpo

Off-Policy			sb3_ddpg	sbx_ddpg		skrl_ddpg
			sb3_td3	sbx_td3		skrl_td3
			sb3_sac	sbx_sac		skrl_sac
			sb3_crossq	sbx_crossq
			sb3_tqc	sbx_tqc

Evolutionary			sb3_ars
						skrl_cem

Imitation-based						skrl_amp

Note: ACME is intentionally absent from SRB algorithm choices. SRB integrations run in the same Python process as Isaac Sim, while current ACME dependencies do not resolve in the Python 3.12 Docker runtime used by SRB.

Known gap — skrl: skrl_sac, skrl_ppo_rnn and skrl_amp are not usable. SquashedGaussianMixin, SequencedMemory and PPO_RNN exist in no released skrl (checked 1.4.3 and 2.1.0), and nothing passes a motion dataset to skrl’s AMP agent, so its discriminator never sees the reference motions. Each needs code in SRB, not a config change. The remaining skrl entries are written against the pinned release and are exercised by tests/unit/test_skrl_hyperparams.py, which also records the three gaps — but note that no skrl profile has a trained policy behind it in this repository, so treat their hyperparameters as untuned starting points.

This time, you can train another agent using an algorithm of your choice:

srb agent train --headless --algo <ALGO> --env landing env.num_envs=1024

Hint: Use --headless mode with more parallel environments for faster convergence.

4. Monitor Training Progress

While training, you might be interested in monitoring the progress and comparing different runs through a visual interface. Local run artifacts and TensorBoard are the accepted defaults; W&B is an optional explicit opt-in and is not required for acceptance evidence. TensorBoard logs are saved in the space_robotics_bench/logs directory. You can start TensorBoard to visualize the training progress:

tensorboard --logdir ./logs --bind_all

You can enable Weights & Biases (wandb) logging with framework-specific flags:

DreamerV3: srb agent train ... 'agent.logger.outputs=[tensorboard,wandb]'
SB3 & SBX: srb agent train ... +agent.wandb=true
skrl: srb agent train ... agent.agent.experiment.wandb=true

Note: Logging to Weights & Biases requires an account and API key.

5. Configure Hyperparameters

Reference: Agent Configuration

The default hyperparameters for all algorithms and environments are available under the space_robotics_bench/hyperparams directory. Similar to the environment configuration, you can adjust the hyperparameters of the selected RL algorithm through Hydra. However, the available hyperparameters and their structure is specific to each framework and algorithm.

Here are some examples (consult hyperparameter configs for more details):

srb agent train --algo dreamer      agent.run.train_ratio=128  ...
srb agent train --algo sb3_ppo      agent.gamma=0.99           ...
srb agent train --algo sbx_sac      agent.learning_rate=0.0002 ...
srb agent train --algo skrl_ppo     agent.models.separate=True ...

Hyperparameter Tuning

srb agent tune optimizes reinforcement-learning hyperparameters with Optuna. It answers two needs at once:

One tuned set per algorithm, shared across all tasks — the default (matches the global hyperparams/<framework>/<algo>.yaml files), with a separate set for state-based vs visual observations.
Optional per-task overrides tuned for a single task, which take precedence for that task only.

Install the optional dependency first:

uv sync --extra tune

How it works

The tuner is Isaac-free. It owns an Optuna study and, for each trial, spawns one srb agent train subprocess:

study.ask() samples a hyperparameter set from the algorithm’s search space.
The driver writes a single trial_cfg.yaml — a complete agent: section plus a minimal env: section (seed, num_envs) — and launches srb agent train --cfg trial_cfg.yaml --headless in its own process group.
A lightweight reporter wrapper inside the child writes episode statistics to a JSONL file; the driver tails it, counting total environment steps.
Smoothed return is reported at 20 fixed rungs; a MedianPruner kills unpromising trials (killpg on the whole group). Crashes and timeouts FAIL the trial and divergence (NaN) records it as PRUNED — neither takes down the study.
study.tell() records the tail-mean of the objective over the last quarter of training.

Because each trial is a separate process, one bad hyperparameter set (VRAM blow-up, NaN physics) cannot corrupt the study, and multiple workers/nodes can share one journal file.

Config layering

Configs resolve task-specific → global, in the requested observation modality:

hyperparams/
  skrl/ppo.yaml                     # L0  — global, state observations (exists today)
  skrl/ppo_visual.yaml              # L0v — global, visual observations
  task/<task>/skrl/ppo.yaml         # L1  — task-specific, state
  task/<task>/skrl/ppo_visual.yaml  # L1v — task-specific, visual

--obs {state,visual} (on train, eval, and tune) selects the modality. It only rewrites the config entry key and splits the logdir workflow name (…/<algo>/ vs …/<algo>-visual/) so state and visual checkpoints never cross-load — the algorithm label is unchanged. Visual studies additionally require sensor profiles and arrive in a later phase; until then --obs visual errors with the file to add.

Study modes

Before committing to a long study, preflight the exact command with srb agent tune check (swap run→check). It resolves the algorithm, search space, and base config, checks your --pins, and dry-assembles one trial doc in memory — no Isaac boot, no study written — so a typo or a missing portfolio baseline fails in milliseconds instead of after the first trial spins up.

Single-task (default) tunes one task and exports to its task-specific slot:

srb agent tune run --env sample_collection --algo skrl_ppo \
  --trial-steps 20_000_000 --n-trials 40 --n-workers 2 --gpus 0,1

Multi-task (portfolio) studies tune one hyperparameter set across a task set and export it to the global slot:

srb agent tune run --tasks sample_collection peg_in_hole landing --algo skrl_ppo \
  --trial-steps 10_000_000 --n-trials 30 --normalize baseline --aggregate median

Each trial trains the sampled set sequentially on every task (one child per task segment); a diverging segment prunes the whole trial, and the running aggregate is reported at each segment so weak candidates are cut between tasks.

Cross-task normalization

Tasks have different reward scales, so raw returns are not directly comparable. --normalize maps each task’s score onto a common scale before --aggregate (median default, or mean/min) combines them:

baseline — (score − floor) / (ref − floor), where floor is a random policy and ref a reference run. Capture both first with srb agent tune baseline --tasks … --algo … --budget <trial-steps>, which writes hyperparams/tune/baselines.yaml. A missing baseline fails fast; a stale one (the source config’s git blob SHA changed since capture) warns.
rank — per-task fractional rank across completed trials; needs no baselines but is non-stationary early, so it is off by default.
none (default) — aggregates raw returns; only meaningful when the tasks already share a scale, and the driver warns for a portfolio study otherwise.

Per-task raw and normalized scores are recorded in each trial’s user_attrs (seg/<task>/raw, seg/<task>/norm).

Picking a trial budget (wall-clock worksheet)

--trial-steps is total environment steps, identical across frameworks. Do not copy the production budget (those yamls train until stopped, e.g. 2e8). Instead pick the step count at which a reference run reaches ~50–70 % of its converged return, then:

hours_per_trial ≈ trial_steps / (env_steps_per_second) / 3600
study_hours     ≈ n_trials × hours_per_trial / n_workers      (pruning cuts this 30–50 %)

Worked example. A task sustaining 40 000 env-steps/s with --trial-steps 20_000_000 → ~8.3 min/trial → 40 trials on 2 workers ≈ 2.8 h. A heavier task at 5 000 env-steps/s → the same budget is ~67 min/trial → ≈ 22 h. Pass --fps to have tune run print this projection, and --max-wall-hours to stop asking new trials past a cap. Start with a 10–20-trial smoke study before committing to 40–150 trials.

Inspecting and exporting

srb agent tune status --study srb-skrl_ppo-state-sample_collection-v1
srb agent tune export --study srb-skrl_ppo-state-sample_collection-v1 --to task --dry-run
srb agent tune export --study srb-skrl_ppo-state-sample_collection-v1 --to task

Before exporting, consider srb agent tune validate --study S --top-k 5 --seeds 4 as a noise arbiter: TPE scores each trial on one seed, so best_trial can be a seed-lucky outlier. validate re-runs the top-K trials on several fresh seeds and re-ranks them by the multi-seed mean, printing the robust winner (and flagging when it differs from best_trial). It is advisory — nothing is written into the study — so you then export --trial <robust winner>.

To attack that same seed noise during the search instead of after it, pass --seeds-per-trial K to run: each trial trains K children on distinct seeds and the sampler is told their mean. Pruning runs on the first replicate only, so bad configs still cost one child while good ones cost ~K×. It trades compute for a cleaner objective; validate trades nothing but only re-ranks the survivors. Use one, the other, or both.

export writes the winning trial’s tuned hyperparameters to the L0/L1 slot with a provenance header (study, trial, value, objective, seeds, date, git SHA, Optuna version). The trial’s tiny budget and forced-off W&B flag are reset to the production defaults. --to defaults to the task slot for single-task studies and the global slot for portfolio studies (--to task on a portfolio study is ambiguous and errors). It refuses to overwrite without --force, and warns about shadowing in both directions: a global export that existing task-specific files will keep overriding, or a task export that shadows the current global set for that task. Adoption is ordinary git review — inspect the diff and commit.

Parallelism and resuming

--n-workers N runs N concurrent trial slots in one driver; --gpus 0,1 round-robins GPUs; child starts are staggered to avoid concurrent Isaac first-boot shader-cache races (the stagger stops automatically once the first child has booted).
Multi-node: run the same tune run command on each node against the shared logs/tune/<study>/journal.log. A ready-to-adapt SLURM array job is in scripts/tune_slurm.sbatch — each array element is one worker sharing the journal; Optuna coordinates trial hand-out with no scheduler integration.
Re-running the same command resumes (until --n-trials finished trials exist). A changed search space, trial budget, objective, obs mode, or task set is refused with a suggestion to bump --version.
Interrupted RUNNING trials are left alone (safe across nodes); clean them up explicitly with srb agent tune repair --study S --stale-hours H.
Pruned trials’ child logdirs (checkpoints, TensorBoard events) are deleted after the prune to keep disk growth bounded; pass --keep-artifacts to keep them. trial_cfg.yaml and metrics.jsonl always remain.

Reproducibility

Every trial is fully reproducible from its logs/tune/<study>/trials/<n>/ directory: the exact trial_cfg.yaml, the seed recorded in it (seed-base + trial × seeds-per-trial + replicate; replicate seeds live in rep-<k>/trial_cfg.yaml), and the child’s own run metadata. Each child’s metadata.json also records tune: {study, trial}, so any checkpoint can be traced back to its trial, and the study directory carries a driver.json (host, pid, start time) identifying the active driver. See Agent Configuration for where tuned files live and how precedence works.

Imitation Learning Workflow

Imitation Learning (IL) closes the loop between expert demonstrations and deployable policies. The Space Robotics Bench provides a unified two-command workflow — collect then learn — that works identically across every SRB environment.

1. Collect Demonstrations

Reference: srb agent collect — Collect Demonstrations

Bootstrap a dataset by teleoperating the task yourself. The --success_only flag discards failed episodes so the dataset stays clean:

srb agent collect --env peg_in_hole --num_episodes 50 --success_only

By default the keyboard is used as the input device. For higher-fidelity data, use a 6-DoF SpaceMouse or an XR headset (see Extended Reality):

srb agent collect --env peg_in_hole --teleop_device spacemouse --success_only

Datasets are written to ${SRB_LOGS_DIR}/<env_id>/collect/<timestamp>/datasets/demos_<timestamp>.hdf5 (HDF5), directly consumable by the learn command.

2. Train an IL Policy

Reference: srb agent learn — Learn from Demonstrations

The srb agent learn command runs offline and does not require Isaac Sim, which makes it fast to iterate on:

srb agent learn --algo robomimic_bc --env peg_in_hole \
  --dataset ${SRB_LOGS_DIR}/peg_in_hole/collect/<timestamp>/datasets/demos_<timestamp>.hdf5

Pass the dataset path explicitly with --dataset: srb agent learn creates a fresh run directory and does not auto-discover datasets from prior collect/ runs. The RNN variant works the same way:

srb agent learn --algo robomimic_bc_rnn --env peg_in_hole \
  --dataset ${SRB_LOGS_DIR}/peg_in_hole/collect/<timestamp>/datasets/demos_<timestamp>.hdf5

3. Evaluate the Policy

Reference: srb agent eval — Evaluate Agent

Policies trained by learn share the SRB checkpoint convention, so evaluation is just another srb agent eval invocation:

srb agent eval --algo robomimic_bc --env peg_in_hole env.num_envs=1

Robomimic runs one environment. The integration wrapper and the collection path both reject num_envs != 1 (integration.robomimic.vector_env.unsupported), so vectorized evaluation, collection, and training are unavailable for robomimic_* algorithms.

4. (Optional) Iterate

IL datasets are rarely sufficient on the first try. Two useful follow-ups:

Augment with Policy Rollouts

Use a partially trained policy to generate additional demonstrations — often called DAgger-style data augmentation. Filter by success to keep the dataset clean:

srb agent collect --algo robomimic_bc --env peg_in_hole \
  --num_episodes 200 --success_only

Rerun srb agent learn on the new demonstrations. Since srb agent learn consumes a single --dataset, merge the collection runs’ HDF5 files first to train on their union.

Combine with RL

IL provides a good starting point but rarely produces a policy that generalizes perfectly to unseen conditions. RL checkpoints cannot be warm-started directly from IL checkpoints (the frameworks use incompatible checkpoint formats), but the two paradigms still compose well:

Use the IL policy to collect additional demonstrations (see above) while training an RL agent from scratch on the same task.
Resume an interrupted RL run with srb agent train --continue_training, or pass a same-framework checkpoint explicitly via --model.

5. Deploy to Hardware

Reference: Sim-to-Real Transfer

IL-trained policies go through the same sim-to-real pipeline as RL policies. See the Sim-to-Real Transfer workflow for the real_agent gen → real_agent eval bridge.

Sim-to-Real Transfer

The Space Robotics Bench provides a streamlined workflow for deploying agents trained in simulation directly onto physical hardware. This is managed through the real_agent command-line interface, which runs a hardware-interfacing equivalent of a simulated environment. This allows various workflows, including the deployment of a trained RL policy, to be executed on a real robot with minimal changes.

1. Train your Agent in Simulation

Reference: Reinforcement Learning Workflow

The first step is to train a policy in simulation. The goal is to produce a stable policy checkpoint. All RL frameworks and algorithms integrated into SRB are supported by this sim-to-real workflow.

Let’s train a Dreamer agent for the waypoint_navigation task with the Leo Rover on 512 parallel environments. Since we are deploying in an on-Earth facility, we will also specify the earth gravity setting. (For the best results, it is highly recommended to use a combination of Domain Randomization and Procedural Generation over the course of the training. This creates a more robust agent that is better prepared for the complexities of the real world.)

srb agent train --headless --algo dreamer --env waypoint_navigation env.robot=leo_rover env.num_envs=512 env.domain=earth

2. Generate Sim-to-Real Bridge

Reference: srb real_agent gen — Generate Sim-to-Real Bridge

This key step creates the bridge between the simulation and the real world. The srb real_agent gen command inspects a simulated Gymnasium environment and automatically writes a lightweight, real-world counterpart that does not depend on the simulation backend.

You can specify which default HardwareInterface modules your robot uses via the --hardware flag. These are the drivers that communicate with your robot’s software, e.g., via ROS 2. For the Leo Rover, we will need an interface to send velocity commands (ros_cmd_vel) and one to receive pose information (ros_tf). Furthermore, we will use the ros_mw interface to expose ROS 2 service calls for pausing and resuming the agent. It is important to specify the robot here so that its parameters, such as action scaling, can be correctly extracted for the real-world environment.

Tip: To see every available hardware interface ID, run srb real_agent gen --help, or browse the sim_to_real/hardware directory.

srb real_agent gen --env waypoint_navigation env.robot=leo_rover --hardware ros_cmd_vel ros_tf ros_mw

This command launches a temporary headless SRB session. It loads the environment, inspects its APIs, and then writes a new Python file inside the sim_to_real/env directory. This Python file defines a RealEnv class and registers it in Gymnasium under the srb_real/ namespace.

3. Deploy and Evaluate on Hardware

With the bridge generated, you can deploy the agent to your robot. The real_agent command does not launch a simulation. Instead, it runs the generated RealEnv, which connects directly to your hardware.

To evaluate the policy trained in the first step, run the following command.

srb real_agent eval --env waypoint_navigation --algo dreamer

The RealEnv will instantiate the ros_cmd_vel, ros_tf, and ros_mw interfaces. When the policy produces an action, the environment routes it to the RosCmdVelInterface, which publishes it as a ROS 2 message. It then gets the latest pose from the RosTfInterface to use as an observation for the policy’s next step.

4. Advanced Workflows and Use Cases

The real_agent tool is not just for evaluation. It enables several powerful workflows for research and development.

Debugging with Zero and Random Agents

Before deploying a fully autonomous policy, you can use the zero and rand agents to quickly test your hardware setup. The zero agent does nothing, while the rand agent sends random actions to the robot.

srb real_agent zero --env waypoint_navigation

srb real_agent rand --env waypoint_navigation

Fine-Tuning on Real Data

Note: While SRB supports fine-tuning on real data via the srb real_agent train --continue command, this workflow is still under active development and should be considered experimental.

5. Additional Task Examples

The same workflow generalizes to mobile-manipulation and locomotion tasks. The following subsections show representative deployment commands for the excavation and terrain landscaping tasks, along with notes on expected robot setups, hardware interfaces, and evaluation metrics to monitor.

5.1 Excavation

The excavation task is designed for a Husky + Kinova mobile manipulator equipped with a scoop end-effector payload. Two variants are supported, depending on the motion-generation stack used on the real robot.

OSC (operational-space control) variant:

srb real_agent eval --env excavation --algo skrl_ppo \
  --hardware ros_depth_heightmap ros_tf ros_imu ros_kortex_cartesian ros_mw

skrl_ppo_rnn remains a reserved, fail-closed label because SRB has no recurrent model, sequence-memory, or hidden-state reset contract for the pinned SKRL runtime. The command above illustrates adapter wiring only; it is not hardware-readiness evidence.

The hardware interfaces used here:

ros_depth_heightmap — Subscribes to a RealSense depth stream and projects it into a local heightmap grid.
ros_tf — Receives the base pose from an external mocap/OptiTrack source.
ros_imu — Consumes IMU feedback for base orientation and angular rates.
ros_kortex_cartesian — Drives the Kinova arm via Cartesian admittance and exposes FK and contact-force readings.
ros_mw — Provides middleware service calls for pausing and resuming the agent.

Joint-velocity variant:

srb real_agent eval --env excavation --algo skrl_ppo_rnn \
  --hardware ros_depth_heightmap ros_tf ros_imu ros_kortex_joint_vel ros_mw

Additional interfaces relative to the OSC variant:

ros_kortex_joint_vel — Sends joint-velocity commands to the Kinova arm. Use this variant when an external motion-generation layer (e.g. NVIDIA fabrics) is responsible for translating policy outputs into reactive joint targets.

Evaluation metrics the operator should watch during rollouts: success_rate, terminal_reward, particle_count_in_scoop_volume, and action_smoothness.

5.2 Terrain Landscaping

The canonical landscaping deployment is RaphRover + prismatic-velocity RaphShovel with one external heightmap source. It has no onboard depth camera, no IMU observation, no blade-pitch channel (measured or synthetic), no previous-action feedback, and no Leo Rover profile. The legacy Leo adapter stack is kept only under the separately named hyperparams/validation/terrain_landscaping_leo.yaml profile; it does not share the canonical task IDs and is not generated by the commands below.

There are two canonical task IDs, and the generated modules are distinct:

Task ID	Target manifest	Generated module
`terrain_landscaping_crater`	fixed crater manifest, hash pinned in the module	`srb/interfaces/sim_to_real/env/terrain_landscaping_crater.py`
`terrain_landscaping`	none yet — declares `REQUIRES_RUNTIME_TARGET_MANIFEST = True`	`srb/interfaces/sim_to_real/env/terrain_landscaping.py`

The general task must never be relabelled with the crater hash. It now owns a hashed per-episode mission manifest of its own, so RealEnv construction still demands one at deployment time (REQUIRES_RUNTIME_TARGET_MANIFEST) rather than borrowing the crater’s.

# Regenerate the checked-in bridge modules (writes the module in place)
srb real_agent gen --env terrain_landscaping_crater
srb real_agent gen --env terrain_landscaping

# Verify the checked-in modules are byte-identical to a fresh render
srb real_agent gen --env terrain_landscaping_crater --check
srb real_agent gen --env terrain_landscaping --check

Both commands run unqualified. The general task previously needed a particle-height override, because plain gen --env terrain_landscaping aborted during env.reset() with ExternalHeightmapFrameError: its particle bed spilled off an undersized containment surface and heaped past the D5 elevation envelope. That defect is fixed in the task itself, so no override belongs in these commands and none must be reintroduced.

--check is read-only: it renders and formats a candidate, byte-compares it with the checked-in module, prints a unified diff, and exits non-zero on drift without touching the file. It needs the repo’s pinned formatter (ruff) on PATH — without one it refuses to compare rather than report drift that an unformatted candidate would manufacture. srb real_agent gen --env ALL --check runs that gate across every cached environment and exits non-zero if any has drifted, or if the environment cache is empty (a gate that inspected nothing is not a pass). Generated modules whose task is no longer registered are absent from the cache and therefore outside the batch’s reach.

Generated schema. Generation derives the module from the frozen IO contract through the typed env.srb_spec seam — never from one sampled step return — and refuses to emit np.finfo dtype-extrema bounds:

3 actions, all normalized velocities: robot/cmd_vel on [0:2] (linear, angular) and payload/joint_vel on [2:3].
7 actor observation leaves with finite physical bounds: proprio_dyn/heightmap_current_global, proprio_dyn/heightmap_target_global, proprio_dyn/heightmap_current_local, proprio_dyn/heightmap_target_local, proprio/mission_pose, proprio/base_velocity, and proprio_dyn/heightmap_age_s — 1287 float32 in contract order.
The assembled SINGLE_OBSERVATION_SPACE is four map outputs plus one packed vector; that is an assembly of the seven leaves, not seven top-level outputs.
Class constants ENV_ID, IO_SCHEMA_FINGERPRINT, TARGET_MANIFEST_SHA256, REQUIRES_RUNTIME_TARGET_MANIFEST, and DEPLOYMENT_REQUIREMENTS record the deployment identity. Keep target_map_sha256, reset_layout_sha256, manifest_sha256, and io_schema_fingerprint distinct — they are four different hashes and conflating them hides real drift.

Deployment is gated, and nothing in this repository opens the gate. Constructing the canonical landscaping RealEnv validates the union of the capability tags claimed by the supplied hardware interfaces and capability providers before it acquires a ROS node, starts any hardware, or lets a caller load a policy. All four tags are required:

raph.drive_velocity
raph.shovel_prismatic_velocity
landscaping.external_heightmap_batch
landscaping.observable_task_evaluator

Exactly one shipped component claims a tag: srb.interfaces.sim_to_real.validation.landscaping.LandscapingRealEvaluator claims landscaping.observable_task_evaluator. Pass it as RealEnv(..., task_evaluator=...); it grades the accepted external frames against the verified target manifest, owns the reward, and refuses a policy whose model artifact declares a different task id, IO-schema fingerprint, target manifest, reset layout, or normalization policy — before ROS acquisition, hardware start, or inference.

The three remaining tags have no claimant, so construction still raises a typed DeploymentNotReadyError listing the exact missing tags. That is the intended state: the Raph ROS drive/shovel commands and the live external mapper are operator dependencies that must be recorded from the lab first — command topic and message type, unit and sign convention, saturation behaviour, watchdog/timeout semantics, acknowledgement mechanism, and measured joint-state feedback evidence.

Note: Simulation and fake-adapter tests do not prove Raph physical-hardware readiness. Nothing below the gate above has been validated against real hardware.

Real Raph validation is unavailable, and stays unavailable until the external dependencies land. No hardware evidence of any kind exists for any landscaping task. The hardware-free half is complete — the observable task evaluator, the five session-abort outcomes, the horizon-truncation semantics, the model-artifact refusal, and a crater validation spec that resolves its real environment and fails closed. The live half was never started: no live Raph drive/shovel or mapper contract has been recorded, and no lab run exists. Passing every simulation gate on this page would still not imply hardware readiness.

The canonical validation spec is hyperparams/validation/terrain_landscaping_crater.yaml. It declares no hardware, so running it fails closed on the capability preflight (exit 4) until the live adapters exist. Its observable metrics are success_rate, final_grading_mae_m, final_fraction_within_tolerance, action_smoothness, and map_age_s; the final step’s info additionally carries the diagnostic central-ROI and local MAE, sustained success, the four distinct hashes, and the calibration version.

For the full readiness picture — which learners are usable, what the observation contract actually is, and which gates are blocked — see Terrain Landscaping Training Readiness. In particular: TD-MPC2 is disabled for both landscaping task IDs and fails closed before any expensive side effect, and no learned landscaping policy is claimed, so a real crater session driven by a current policy would be expected to report a final grading MAE close to the zero-action baseline and success_rate = 0. The harness would report that as exit 1, correctly; A8a proves the reporting is truthful, not that anything can pass.

6. Validation Workflow

Advanced / reference. Sections 1–3 above are all you need to deploy and evaluate a policy. This section documents the optional structured-validation harness (release gating, drift tracking, dashboards) and is reference-level detail — skip it unless you are setting up repeatable release validation.

Once a checkpoint is deployed via srb real_agent eval, you can run a structured validation session that produces a machine-readable pass/fail, multi-modal telemetry (rerun.io, JSONL, optional W&B), and a static dashboard tracking drift across releases.

6.1 Capture a sim baseline

The sim baseline is captured once per checkpoint by reusing Isaac Lab’s existing RerunVisualizer:

srb agent eval --env excavation --algo skrl_ppo_rnn --model <ckpt> \
    env.num_envs=1

The checkpoint is passed with --model (not --checkpoint), and the parallel env count is a Hydra override (env.num_envs=). The episode count, .rrd baseline recording (rerun.record_to_rrd), and W&B logging are configured through the validation spec YAML (see §6.2) rather than as srb agent eval flags. The W&B summary scalars (success_rate, terminal_reward, action_smoothness, etc.) are referenced later by the validation harness via the run id; the .rrd file enables twin-replay debugging.

6.2 Validation spec YAML

The validation spec is the persistent Interface between an operator, a generated RealEnv, and the validation harness. CLI flags only override a few run-time fields; the YAML keeps the task, policy, hardware adapters, criteria, telemetry, baseline, and drift contract together.

task: excavation
algo: skrl_ppo_rnn
checkpoint: REQUIRED_AT_RUNTIME
hardware:
  - ros_depth_heightmap
  - ros_tf
  - ros_imu
  - ros_kortex_cartesian
  - ros_mw
metrics_to_track:
  - success_rate
  - terminal_reward
  - action_smoothness
thresholds:
  success_rate_min: 0.5
  sim_real_ratio_min: 0.7
  action_smoothness_max_ratio: 1.5
  crash_rate_max: 0.0
  drift_z_max: 3.0
sim_baseline:
  wandb_run_id: entity/project/run-id
  local_rrd: baselines/excavation/<sha>.rrd
rerun:
  app_id: srb-validation
  web_port: 9090
n_episodes: 10
drift_window: 5
storage_root: logs/real_validation
notes: excavation release gate

Top-level fields:

Field	Type	Required	Default	Purpose
`spec_id`	string or null	no	`null`	Legacy compatibility identifier from older spec sources.
`task`	string	yes	REQUIRED	RealEnv task id resolved as `srb_real/<task>`.
`algo`	string	yes	REQUIRED	Policy adapter algorithm slug used to load the checkpoint.
`checkpoint`	string	yes	REQUIRED	Raw checkpoint or SRB model artifact path.
`hardware`	list[string]	yes	REQUIRED	Hardware Interface ids instantiated by the generated RealEnv.
`metrics_to_track`	list[string]	yes	REQUIRED	Metric names included in cross-session drift checks.
`n_episodes`	integer	no	`10`	Number of hardware rollout episodes.
`max_episode_seconds`	float seconds	no	`120.0`	Per-episode wall-clock safety cap.
`thresholds`	object	no	See `thresholds.*`	Criterion (a) threshold and drift limits.
`rerun`	object	no	See `rerun.*`	Live viewer and `.rrd` telemetry settings.
`sim_baseline`	object	no	See `sim_baseline.*`	Simulation baseline references used for sim/real ratios.
`drift_window`	integer sessions	no	`5`	Previous sessions used for z-score drift evaluation.
`storage_root`	path	no	`logs/real_validation`	Root directory for validation session artifacts.
`fault_injection`	object or null	no	`null`	Optional dry-run fault toggles for harness smoke tests.
`notes`	string	no	`""`	Free-form operator note copied into the spec snapshot.

Threshold fields:

Field	Type	Required	Default	Purpose
`thresholds.success_rate_min`	float	no	`0.5`	Minimum real-hardware success rate.
`thresholds.sim_real_ratio_min`	float	no	`0.7`	Minimum real/sim metric ratio (higher-is-better metrics).
`thresholds.lower_is_better_metrics`	list[string]	no	`[]`	Metric names where lower real values are better (error/loss); their real/sim ratio is capped above at 1/sim_real_ratio_min instead of floored. Error-like names are auto-detected; list extra ones here.
`thresholds.action_smoothness_max_ratio`	float	no	`1.5`	Maximum real/sim action-smoothness ratio.
`thresholds.crash_rate_max`	float	no	`0.0`	Maximum allowed crash or safety-event rate.
`thresholds.drift_z_max`	float	no	`3.0`	Maximum z-score for tracked metrics across recent sessions.

Rerun fields:

Field	Type	Required	Default	Purpose
`rerun.app_id`	string	no	`srb-validation`	Rerun application id for live validation telemetry.
`rerun.web_port`	integer	no	`9090`	Local web viewer port.
`rerun.grpc_port`	integer	no	`9876`	Rerun gRPC port.
`rerun.bind_address`	string or null	no	`0.0.0.0`	Viewer bind address.
`rerun.keep_historical_data`	bool	no	`true`	Keep historical data visible in the Rerun viewer.
`rerun.keep_scalar_history`	bool	no	`true`	Keep scalar time-series history in the viewer.
`rerun.record_to_rrd`	bool	no	`true`	Persist the session recording as `rollout.rrd`.

Simulation baseline fields:

Field	Type	Required	Default	Purpose
`sim_baseline.wandb_run_id`	string or null	no	`null`	W&B run id containing simulation summary scalars.
`sim_baseline.local_rrd`	path or null	no	`null`	Local simulation `.rrd` baseline for twin replay.
`sim_baseline.metrics_json`	path or null	no	`null`	Local JSON baseline metrics file.

Fault-injection fields:

Field	Type	Required	Default	Purpose
`fault_injection.nan_policy`	bool	no	`false`	Force a policy-NaN validation failure path.
`fault_injection.missing_baseline`	bool	no	`false`	Force missing-baseline handling.
`fault_injection.hardware_event`	bool	no	`false`	Force hardware-event handling.

6.3 Run validation on the real robot

srb real_agent validate \
    --spec hyperparams/validation/excavation.yaml \
    --checkpoint <real-or-shared-ckpt> \
    --episodes 10 \
    --storage-root /tmp/srb_validation

Validation accepts --spec, --checkpoint, --episodes, --wandb, --no-tensorboard, --no-rerun-web, --storage-root, --post-status, and --dry-run. Local artifacts and TensorBoard remain the default; W&B requires explicit opt-in.

--dry-run is explicitly non-hardware and never passes. It skips environment and policy instantiation, then finalizes with pass_overall: false, hardware_evidence: false, exit_code: 4, and metrics.n_episodes: 0. Its three criteria carry status skipped, and the badge is red with a non-hardware evidence message. The session is not recorded in the drift store. Treat it as a wiring smoke artifact only; it is not real-data or hardware evidence and cannot be used as a validation pass.

The harness:

Spawns a rerun web viewer at http://localhost:9090 for live in-session inspection.
Runs <--episodes> rollouts on the real robot via the RealEnv generated earlier with srb real_agent gen.
Writes the validation session and storage-root artifacts described in the next section.
Compares against the sim baseline using three criteria:
- (a) Threshold: success rate ≥ minimum, sim/real ratios in bounds. The success-rate floor is evaluated even when no sim baseline is configured — only the sim/real ratio half is skipped — so a session without a single successful episode never exits 0.
- (b) Crash: no policy_nan, hardware_event, disk_full terminations, and no safety-relevant session abort (sensor_stale, sensor_incomplete, calibration_mismatch, manifest_mismatch)
- (c) Drift: z-score of all tracked metrics ≤ thresholds.drift_z_max over the last K sessions

Episodes that end at the environment’s configured horizon are recorded as time_limit_truncation (a truncation carrying the final observable metrics), never as the harness’s wall-clock timeout. Out-of-MDP session aborts get their own term reasons and per-reason counts in the session metrics: sensor_stale, sensor_incomplete, calibration_mismatch, manifest_mismatch, and operator_abort. None of them is a task termination or a truncation; an operator stop fails the success-rate floor rather than the crash criterion.

Exits with the validation result code:

Exit code	Name	Criterion	Meaning
`0`	`pass`	`all criteria pass`	Validation met threshold, crash, and drift criteria.
`1`	`threshold_fail`	`criterion_a`	Threshold metrics failed, such as success rate or sim/real ratio.
`2`	`crash_fail`	`criterion_b`	A crash or safety-relevant termination was observed.
`3`	`drift_fail`	`criterion_c`	Tracked metrics drifted beyond the configured z-score limit.
`4`	`preflight_fail`	`preflight`	Validation could not start rollout, for example due to missing generated environment code, missing hardware wiring, an incompatible checkpoint, or an explicit non-hardware dry-run.

A validation session always runs the generated srb_real/<task> environment; the command registers those ids itself before constructing one. If the real environment cannot be resolved, validation raises — it never substitutes the srb/<task> Isaac simulation task, because the session would still be written up as a real-validation report.

Which shipped specs can actually run. Removing that simulation fallback was correct — it was writing simulation results up as hardware proof — but it also means most shipped specs now stop at pre-flight. Only a spec whose task has a generated module under srb/interfaces/sim_to_real/env/ is runnable:

Spec (`hyperparams/validation/`)	`srb_real/<task>` generated?	Result of `validate --spec`
`excavation.yaml`	yes	runs
`formation_following.yaml`	yes	runs
`terrain_landscaping_crater.yaml`	yes	runs
`beam_transport.yaml`	no	exits 4 (pre-flight); `srb real_agent gen --env beam_transport` first
`formation_following_decentralized.yaml`	no	exits 4; no generated single-rover decentralized env
`spacewalk.yaml`	no	exits 4; `gen` cannot help — no spacewalk deployment profile exists
`spacewalk_eva.yaml`	no	exits 4; same, no deployment profile
`spacewalk_iva.yaml`	no	exits 4; same, no deployment profile
`terrain_landscaping_leo.yaml`	no	exits 4; the legacy Leo bridge is no longer generated

The pre-flight error message suggests srb real_agent gen. That remedy is correct only for the rows above where a deployment profile exists (srb/interfaces/sim_to_real/deploy_profiles/generation.py); for the three spacewalk specs there is no profile at all, so nothing can be generated for them today. Each non-runnable spec carries a # NOT RUNNABLE header saying so. No shipped spec is claimed to have been validated on hardware.

6.4 Validation artifacts

Validation writes one session directory plus two storage-root artifacts:

logs/real_validation/
+-- _index.jsonl
+-- index.html
+-- <task>/
    +-- <uuid>_<utc-timestamp>/
        +-- current.json
        +-- generations/
            +-- <generation-uuid>/
                +-- spec.yaml
                +-- episodes.jsonl
                +-- rollout.rrd
                +-- tensorboard/
                +-- summary.json
                +-- report.md
                +-- badge.json

Session artifacts:

Path	Scope	Written by	Produced when	Purpose
`spec.yaml`	session directory	`write_spec_snapshot`	pre-flight	Validation spec snapshot required by rebuild-summary.
`episodes.jsonl`	session directory	`Telemetry`	rollout	Append-only step, event, and episode-end telemetry used for metrics.
`rollout.rrd`	session directory	`Telemetry`	when `rerun.record_to_rrd=true` and Rerun starts	Rerun recording used for live inspection and twin replay.
`tensorboard/`	session directory	`Telemetry`	when TensorBoard logging is enabled and available	Optional scalar event logs for local dashboard inspection.
`summary.json`	session directory	`write_session_report`	post-session finalization or rebuild-summary	Machine-readable pass/fail result and criterion payload.
`report.md`	session directory	`write_session_report`	post-session finalization or rebuild-summary	Human-readable validation session report.
`badge.json`	session directory	`write_session_report`	post-session finalization or rebuild-summary	Shields.io endpoint payload for validation status display.

Storage-root artifacts:

Path	Scope	Written by	Produced when	Purpose
`_index.jsonl`	storage root	`DriftStore`	post-session finalization	Append-only session index used for drift history and dashboard rows.
`index.html`	storage root	`write_validation_index`	post-session finalization	Static cross-session dashboard for pass/fail state and metrics.

6.5 Cross-session dashboard

A static HTML dashboard at logs/real_validation/index.html is regenerated on every run. Open it in a browser to see all sessions, their pass/fail state, and trend charts per (task, algo, metric).

6.6 Twin replay (debugging)

When a session fails, compare it against the sim baseline:

srb real_agent replay-twin \
    --session logs/real_validation/excavation/<uuid>_<ts>/generations/<generation-uuid> \
    --baseline baselines/excavation/<sha>.rrd

The rerun viewer opens with sim ghost (transparent) overlaid on real (solid), time-aligned by episode index.

6.7 Recovering a partial session

If a session crashes mid-rollout (disk full, network loss, etc.), summary.json will be missing. Rebuild it from spec.yaml and episodes.jsonl:

srb real_agent rebuild-summary logs/real_validation/excavation/<uuid>_<ts>/generations/<generation-uuid>

6.8 Safety reminder

The validation harness observes safety events (estop, joint-limit violations, contact-force spikes) but does not enforce them. Physical safety remains the operator’s and hardware bring-up’s responsibility — the harness records these events and counts them toward criterion (b), but does not replace deadman switches, joint-limit watchdogs, or estop circuitry.

7. Creating Custom Hardware Interfaces

You can easily support custom sensors or actuators. To create a new interface, add a new Python file in the sim_to_real/hardware directory. Your new class should inherit from the HardwareInterface base class.

You will need to implement a few key methods:

start to initialize your hardware connection.
apply_action to send commands.
observation to get sensor data.
close to clean up connections.

The system will discover your new interface automatically, making it available to the --hardware flag.

Terrain Landscaping — Training Readiness

Read this first. This page records what the three landscaping tasks actually prove, and what they do not. Several readiness gates are blocked or unmeasured. No learned policy is claimed for any of them, TD-MPC2 landscaping is deliberately disabled, and no Raph hardware evidence exists. Nothing on this page should be read as a recommendation to start a long training run.

The machine-readable version of everything below is docs/superpowers/evidence/terrain_landscaping/readiness.json in this repository (it is not published on the website, and the path is deliberately not linked to a branch that may not carry it yet). Each gate cites the reviewed evidence record it reduces, in docs/superpowers/evidence/terrain_landscaping/.

1. Three separate tasks, and no curriculum

There are three landscaping task IDs. They are separate tasks with separate readiness claims, not stages of one progression.

	`terrain_landscaping_crater` (simple)	`terrain_landscaping_mound`	`terrain_landscaping` (general)
Purpose	one deterministic, fixed problem — the first training and deployment-test environment	a second deterministic baseline, added because the crater task’s grading metric is not reachable by this hardware	the full task: an arbitrary hashed mission per episode
Target	the checked-in analytic flat-bed manifest `crater_target_manifest.json`, hash-pinned	the checked-in `mound_target_manifest.json` — the same flat analytic bed, its own reset layout `mound_pile_v1`	a per-episode mass-balanced deformation of the flat reset bed, hashed per mission
Episode	300 steps (30 s)	300 steps (30 s)	600 steps (60 s)
Grading cells	2398	2398	6237
Reset	one fixed crater + one loose pile, fixed rover pose	one 0.24 m conical pile on a flat bed, no crater, fixed rover pose on the pile centreline	flat particle bed, mission drawn from a deterministic seed stream
Gate metric	observable grading MAE, margin 2.0 mm	excess volume above target, margin 1.75 mL	observable grading MAE, margin 2.0 mm

No curriculum exists anywhere in this stack. There is no automatic difficulty progression, no curriculum scheduler, no implicit switching from the crater task to the general task, and no claim that a crater-trained model solves general landscaping. The general task’s module is checked by an AST-based test that bans the identifiers curriculum, difficulty, task_crater, and the crater manifest loaders from its code, so the separation cannot silently erode.

A result on one task is not evidence for any other, in any direction.

2. What is proven, and what is not

Gate	Subject	Status
G0	IO contract, manifests, four hashes, no curriculum	passed
G1	particle physics and readback (Isaac Sim 6.0.1 specific)	passed
G2	simple task is a reachable stationary-reward problem	blocked — actuation
G3	external-heightmap observation seam	passed
G4	Dreamer integration, strict config, checkpoint semantics	passed (reliability caveat resolved 2026-08-07)
G5	Dreamer learnability on the simple task	blocked — measured: the model gate fails its margin
G6	reusable one-environment training	blocked — on G5 and on its own memory clause
G6-M	batched (`num_envs > 1`) training	blocked — refused fail-closed
G7	TD-MPC2 on landscaping	blocked by design (D13)
G8	sim-to-real on real Raph	blocked — no hardware evidence exists
G9	general task	blocked — reachability, and no learner evidence
G2-MOUND	mound task is a reachable problem on its volume metric	passed with a caveat — a 1-in-9 per-seed flake
G4-MOUND	Dreamer integration on the mound task	passed with a scope caveat
G5-MOUND	Dreamer learnability on the mound task	blocked — measured: the model gate fails its margin
G6-MOUND	reusable one-environment training, mound task	blocked — on G5-MOUND and on its own memory clause

The claim “one-environment simulation-training-ready for Dreamer” requires G0–G6. G2, G5 and G6 are blocked, so that claim is not made. The mound-task rows do not change that: G5-MOUND is blocked too, and no learned policy is claimed for any landscaping task.

G6 is blocked twice over, and unblocking G5 would not unblock it. Its randomized-profile smoke clause is gated on the G5 model gate, but its resource clause is blocked on its own measurement: peak host RSS over the canary campaign reached 23.33 GiB — 97.2 % of the declared 24 GiB limit — on a run that had only reached ~3,300 steps of a 10,000-step budget. Nothing beyond 10,000 steps was ever measured, and a declared limit may not be raised after a failing run, so the 100,000-step extension’s memory feasibility is unmeasured and on that one peak looks doubtful.

2026-08-07 re-measure: that 23.33 GiB peak came from an aborted run on the pre-fix geometry. On the fixed containment, six completed 10,000-step canaries peak at 15.9–17.6 GiB (66–73 % of the limit) with a steady-state plateau near 15.5 GiB and an essentially flat last-half slope (+0.02–0.03 GiB/h); on-disk replay is 24–26 MB against the 2 GiB limit. The projection to the 100,000-step extension is comfortable, but the clause stays blocked on its own rule: nothing beyond 10,000 steps has been measured, and the one permitted extension is deliberately unspent while G2 stands. The three seeded randomized-profile smokes have now each completed first-try (early stability evidence, recorded with a scope caveat — the plan orders them after the model gate passes).

G2 and G9 — the environment-reachability wall

Both tasks fail the same gate clause: a deterministic scripted reference must improve the final observable grading MAE by at least 2.0 mm against both a zero-action and a seeded-random baseline.

Simple task (G2): ten distinct scripted controller architectures were designed and measured. The best reached +0.31 mm. The retuned shipped reference lands between −0.32 mm and +0.24 mm — inside the ±0.3 mm bed-creep noise band of the zero baseline itself.
General task (G9): the cut-to-fill shuttle reference finishes 0.18 mm to 0.37 mm worse than doing nothing on all three declared missions.

Two caveats on the G2 number. Each of the ten architectures ran in its own OS process, and for the same simulator/reset seed the settled start state differs between processes by ~0.23 mm of observable grading MAE — the same order as the −0.49 mm..+0.31 mm spread of the campaign. Which of the ten did best is therefore not resolvable from these numbers; that they all fall ~1.5 mm short of the margin is, because 1.5 mm is far outside that effect. The per-seed rows quoted for the shipped reference are within-process and not exposed.

The evaluator is not the blocker. A teleport control that writes ~10 L of particles directly into the crater bowl moves observable MAE from 0.00915 to 0.00436 — the margin is expressible and the observable evaluator responds to it correctly. The measured mechanism is actuation: grading MAE is L1 so relocation inside the ROI is neutral, the observable map is a top-surface raster so material pushed into the bulk disappears from it, a front prismatic plate can shove but not carry, and every cut-to-fill route damages the graded surface it crosses.

Thresholds were not weakened and no episode budget was shortened. Both gate tests are committed at full strength as xfail(strict=False) so they report the deficit instead of masking it. Resolving this is an operator decision over environment-level remedies (a carrying implement, particles much finer than the blade, longer episodes, a volume-based rather than top-surface quality metric, or geometry that permits deposits without bowl transit).

G5 — blocked, and the canary commands are deliberately not published

As recorded on 2026-08-01, the 10,000-step Dreamer canary could not be completed reliably (since resolved — see the 2026-08-07 re-measure below). Of the 21 seeded canary attempt records that survived on disk for the trees under test, 1 finished its budget; 19 died with

ExternalHeightmapFrameError: mission pose [-2.00…, …, …] outside the operating envelope

killing the training process, and 1 was killed from outside. No declared resource limit was ever breached — every abort has a different cause.

The cause is measured: proprio/mission_pose is declared with low = (-2.0, -2.0, -pi), which is exactly the 4 m bed’s half-extent. The rover’s chassis centre can physically drive to and past the bed edge, so a pose 0.3 mm to 10 mm outside the bound is ordinary. The external source treats it as a frame error and raises out of env.step() rather than routing it through the episode-level safety path, so there is no truncation, no reset, and no checkpoint. The same abort is on record from A3’s scripted controller and hit 1 of 2 attempts of the 1,000-step G4 smoke, so it is not specific to the long budget — a short run is lucky, not safe.

Correction (2026-08-01). The paragraph above understates the defect, and the three remedies below were scoped against the understatement. The bed was never 4 m: PlaneCfg.size is a half-extent (spawn_plane scales a UsdGeom.Plane whose default extent is [-1, 1]², and Plane.setup_extras places the walls at ±size; srb/assets/scenery/terrain.py says so directly). Both landscaping tasks passed the declared 4 m edge length straight through, so the spawned surface was 8 m × 8 m with the 0.3 m containment walls measured at ±4.0 m — 2 m outside the D5 mission-pose envelope and outside the D6 manifest ROI. The rover was not “driving to the bed edge”; it was driving across 6 m × 6 m of bare plane that should not have existed. TaskCfg._verify_bed_fits_its_containment, the guard written to catch exactly this, computed 0.5 * min(bed_size_m) and so returned 2.0 m for a 4.0 m half-extent, which is why it passed.

The task geometry is now corrected (_plane_half_extent), and the guard measures the spawned surface instead of deriving it. No declared value changed — the D5 envelope, the D6/D8 geometry, the reset layout, every manifest hash and the io_schema_fingerprint are untouched; only the simulator now matches what they always declared. Every measured number on this page and in the A2/A3/A6a/A6b/A9 evidence records was produced against the 8 m bed and must be re-measured before it is quoted again.

Follow-up (2026-08-02). Two consequences of the same half-extent confusion were cleaned up. Plane.setup_extras derives the environment grid spacing from the plane half-extent (max(size) + 2.0), which for the corrected bed is 4.0 m — exactly the bed’s own edge length, so neighbouring environments’ containment walls would have landed in the same plane (with the old 8 m bed they overlapped by 2 m). TaskCfg.__post_init__ now pins the spacing to max(bed_size_m) + 2.0. Single-environment runs are unaffected (stack keeps env_spacing = 0.0), and num_envs > 1 remains blocked by the separate particle-transport defect. Independently, the step path no longer pulls the particle cache out of the simulator twice: the coarse LandscapingEpisodeTerrain current map it produced had no consumer, and the D7 external source already polls every environment on every step. Measured at num_envs=1: 2.0 → 1.0 simulator particle reads per step, 86.5 → 85.4 ms mean step time (median unchanged at 84.6 ms).

The one surviving checkpoint was evaluated on 2026-08-01 against all three baselines on all five predeclared evaluation seeds, in a single process per seed. It missed the gate, and was reported as indicative only (1 of the 3 required training seeds):

Statistic	Value	Required
trained median final grading MAE	0.009782 m	—
zero median	0.009436 m	—
random median	0.010276 m	—
scripted-reference median	0.009367 m	—
median gain vs zero (difference of medians)	−0.35 mm	≥ +2.0 mm
median gain vs random (difference of medians)	+0.49 mm	≥ +2.0 mm
median paired per-seed difference vs zero	−0.49 mm	≥ +2.0 mm
median paired per-seed difference vs random	−0.11 mm	≥ +2.0 mm

After 10,000 steps that one trained policy was worse than doing nothing on all five evaluation seeds, and worse than the scripted reference. One training seed at the minimum budget is not evidence about learnability in either direction — which is why the full protocol was re-run once the environment defect was fixed.

Re-measure on the corrected bed (2026-08-07)

The envelope-abort defect is resolved by the containment correction above: two independent three-seed 10,000-step canary sets — six canaries — completed on the first attempt each, with zero envelope aborts, zero resource-limit breaches, zero non-finite metrics, and no retries (the reported set is exactly three seeds run once). The full model gate then executed for the first time: all three final checkpoints, evaluated against all three baselines on all five predeclared evaluation seeds, one process per seed. It fails the margin:

Statistic (3 training seeds, 15 paired rollouts)	Value	Required
trained median final grading MAE	0.009943 m	—
zero median	0.009370 m	—
random median	0.010107 m	—
scripted-reference median	0.009402 m	—
median gain vs zero (difference of medians)	−0.57 mm	≥ +2.0 mm
median gain vs random (difference of medians)	+0.16 mm	≥ +2.0 mm
median paired per-seed difference vs zero	−0.42 mm	≥ +2.0 mm
median paired per-seed difference vs random	+0.27 mm	≥ +2.0 mm

All three trained checkpoints lose to the zero-action baseline on the paired statistic (−0.47 / −0.63 / −0.06 mm per training seed), and beat seeded-random by at most +0.55 mm. Every statistic is far below the 2.0 mm margin — and the scripted reference itself achieves only +0.30 mm against zero (the G2 re-measure), so no controller, scripted or learned, can currently express the margin in a 300-step episode. G5’s blocker is therefore no longer “the canary cannot complete”: it is G2’s actuation deficit, and its remedy is the G2 remedy decision, not further training.

Consequently:

the conditional 100,000-step extension was still not run. Its precondition (all three canaries complete) is now met, but the plan names it the second and last qualification attempt; spending it while G2 bounds every controller below the margin would burn the one permitted attempt on a guaranteed failure. It stays unspent pending the G2 remedy decision;
the three randomized-profile 1,000-step smokes (a G6 clause) have now run — early, with a recorded scope caveat, since the plan orders them after the model gate passes (see the G6 note in section 2);
no short-canary command is documented on this page. The plan admits one only after the gate passes from a clean process; the canaries now complete reliably, but the gate itself fails, so no command is published.

Unblocking G5 was believed to require one of exactly three remedies, each reopening a different binding product decision. That framing is superseded by the correction above: remedy 3 turned out to be a plain geometry bug, and fixing it reopens nothing — the bed is now the size D6 always declared, and the rover’s own wheels stop its centre inside the D5 envelope. The 2026-08-07 re-measure confirms it: the canaries complete reliably on the corrected bed, so none of these remedies is needed any more. They are kept as the historical record of the recorded operator options:

route an out-of-envelope mission_pose through the episode-level safety path (truncate and reset that environment) instead of raising out of env.step() — this reopens D11, which declares the collector step limit the only normal rollout boundary and puts real sensor/safety failures outside the MDP;
widen the declared proprio/mission_pose x/y envelope beyond the terrain half-extent — this reopens D5, and therefore changes the IO-schema fingerprint, the generated RealEnv, and every checkpoint’s contract preflight;
constrain the rover so its centre cannot reach the bed edge — this reopens D8, which gives the target manifest ownership of the source/ROI/guard geometry (and D6 too, but only if the source geometry itself moves; D6 merely asserts that the rover’s mission-pose centre stays inside the central 4 m ROI, which is what this remedy restores).

The mound task — reachable, and still not learned (2026-08-08)

terrain_landscaping_mound exists because of the wall above. The crater task’s gate charges the process’s own wheel and blade damage (~1 mm per active episode) against a top-surface L1 metric, and no push pattern with this blade shaves faster than it churns. The hardware is fixed — the embodiment stays the real RaphRover + RaphShovel — so the task was changed instead: level one regolith mound on a flat bed, and grade excess volume above the target rather than surface MAE. On that measure crest removal counts, ruts below target do not, and wheel churn is not charged.

The mound task is a sibling, not a mutation. The crater task, its manifest, its 2.0 mm margin, and its identity hashes are untouched. The mound carries its own reset layout (mound_pile_v1) and its own manifest hash; the D5 observation contract — the exact 1287-float actor layout and the IO-schema fingerprint — is unchanged, so both tasks speak the same interface.

The margin was frozen before the first gate run: 1.75 mL is the larger of a declared 0.5 mL floor and five times the worst measured within-process zero repeatability (0.35 mL). It has not been edited since, and it will not be. Two measured facts force the protocol that goes with it. The bed settles on its own — 5.5–7.6 mL of excess volume leaves every episode under every policy, including doing nothing. And the settled start state differs between OS processes by up to 3.6 mL for the same seed, dwarfing the ≤0.6 mL within-process spread. So each comparison runs all policies in one process from one settled start, and only within-process differences are graded.

G2-MOUND — the environment is reachable. Three consecutive full gate runs of the five-pass scripted reference passed 8 of 9 gate-seed executions (4/4, 3/4, 4/4), including two complete clean-process passes. The residual 1-in-9 flake is recorded rather than retried away: the harness retries crashes only, never assertion failures. This is the first landscaping reachability gate that passes at all.

G5-MOUND — the learner does not clear the margin. Three 10,000-step canaries completed on the first attempt each (740–771 s, 33 × 301-step episodes, no retries, no resource-limit breaches, no non-finite values), and all three checkpoints were then evaluated against both baselines and the scripted reference on all five predeclared evaluation seeds, one process per seed:

Statistic (3 training seeds, 15 paired rollouts)	Value	Required
trained median final excess volume	0.018822 m³	—
zero median	0.019368 m³	—
random median	0.019182 m³	—
scripted-reference median	0.016760 m³	—
median gain vs zero (difference of medians)	+0.55 mL	≥ +1.75 mL
median gain vs random (difference of medians)	+0.36 mL	≥ +1.75 mL
median paired per-seed difference vs zero	+0.58 mL	≥ +1.75 mL
median paired per-seed difference vs random	+1.05 mL	≥ +1.75 mL

Per training seed, the paired difference vs zero is −0.32 / +1.13 / +1.39 mL: two of the three checkpoints beat both baselines, one is worse than doing nothing, and the scripted reference still leads the trained median by 2.06 mL.

This is a different failure from the crater’s. There, the environment could not express the margin at all — the scripted reference reached +0.30 mm against a 2.0 mm requirement, bounding every controller, learned or scripted. Here the same reference clears the margin, so the environment is demonstrably reachable and what falls short is the learner at a 10,000-step budget. That is a learnability result, not an actuation wall — and it is still a failed gate. No learned mound policy is claimed.

The 100,000-step extension was spent, and it made things worse

On operator instruction the same day, all three seed runs were resumed in place to a total of 100,000 steps — the second and last qualification attempt the plan permits — and re-evaluated under the identical protocol against the identical 1.75 mL margin. The trainings were clean (332 × 301-step episodes each, 94–103 min, 8.1–8.2 GiB peak against the 24 GiB limit, 244–246 MiB replay against the 2 GiB limit, zero breaches, zero retries). The result was not:

Statistic	10,000 steps	100,000 steps	Required
median gain vs zero	+0.55 mL	−0.09 mL	≥ +1.75 mL
median gain vs random	+0.36 mL	−0.01 mL	≥ +1.75 mL
median paired difference vs zero	+0.58 mL	−0.08 mL	≥ +1.75 mL
median paired difference vs random	+1.05 mL	−0.21 mL	≥ +1.75 mL
gap to the scripted reference	+2.06 mL	+3.20 mL	—

Per training seed the paired difference vs zero went −0.32 / +1.13 / +1.39 mL → −0.00 / −0.30 / +1.38 mL: only one checkpoint held its gain, and two now sit at or below doing nothing. A 10× budget did not close the gap — it erased the lead, so the 10,000-step positives read as run-to-run spread rather than an early learning trend. The “needs more steps” hypothesis is measured and refuted for this configuration, and no further extension is permitted.

One measurement is not understood and is flagged rather than smoothed over: the 100,000-step runs peak lower than the 10,000-step canaries (8.1–8.2 GiB against 17.5–17.6 GiB single-process), which is backwards for the longer run with the larger replay. No cause was established.

Steps are not the only budget axis: the replay ratio

Everything above measures the steps axis. The gradient-update budget is a separate one. Dreamer does steps × replay_ratio ÷ (batch_size × batch_length) updates, which for this stack’s batch_size: 8 and batch_length: 32 is steps × ratio ÷ 256. Both mound attempts ran at a replay ratio of 16.0 — half this repository’s own default and 1/32 of the 512 the same defaults file gives excavation, the sibling particle-manipulation task. Nothing recorded why. So the two failed attempts did roughly 625 and 6,084 gradient updates, and the diagnosis in docs/superpowers/evidence/terrain_landscaping/2026-08-12-learner-budget-diagnosis.md measured the consequence directly: the actor sat pinned at maximum entropy, wall clock was nearly all simulation, and the gate therefore evaluated an exploration distribution rather than a learned policy.

Both task profiles now declare 512.0. This changes no gate row and no result on this page. The published attempts were run at 16.0 and stay recorded as run; the raise has not itself been carried through a gate reduction at the canary budget, so nothing here is superseded and no learned policy is claimed. It also means the two failed attempts do not bound what this configuration can do — “more steps” was measured and refuted, “more updates” was never the thing under test.

Consequently:

the conditional 100,000-step extension is spent and failed. Its precondition (all three canaries complete) was met, and the plan permits no second one — so any next attempt is an operator decision about what changes, not another run of the same thing;
the three seeded randomized-profile 1,000-step mound smokes have run and were stable (three full episodes each, zero aborts, zero breaches), early and with the same scope caveat as the crater’s;
no short-canary command is documented on this page for the mound task either, for the same reason: the gate does not pass.

3. The actor observation contract

Both tasks publish the same seven leaves and the same three actions. The learner adapter flattens the seven leaves into exactly 1287 float32 in this order, and nothing else enters replay or the world model:

Slice	Leaf	Shape	Physical bound
`[0:256]`	`proprio_dyn/heightmap_current_global`	`(16, 16)`	`[-0.30, 0.50] m`
`[256:512]`	`proprio_dyn/heightmap_target_global`	`(16, 16)`	`[-0.30, 0.50] m`
`[512:896]`	`proprio_dyn/heightmap_current_local`	`(24, 16)`	`[-0.30, 0.50] m`
`[896:1280]`	`proprio_dyn/heightmap_target_local`	`(24, 16)`	`[-0.30, 0.50] m`
`[1280:1283]`	`proprio/mission_pose`	`(3,)`	`x, y ∈ [-2.0, 2.0] m`, `yaw ∈ [-π, π]`
`[1283:1286]`	`proprio/base_velocity`	`(3,)`	`vx, vy ∈ [-1.5, 1.5] m/s`, `wz ∈ [-4.0, 4.0] rad/s`
`[1286:1287]`	`proprio_dyn/heightmap_age_s`	`(1,)`	`[0.0, 0.5] s`

Elevations are metres relative to the mission datum (the containment base plane, z = 0). The global maps are an area-mean downsample of the 4 m ROI; the local maps sample native cell centres in the rover body frame (+x forward, +y left) at the frame’s own synchronized pose, with bilinear interpolation and no extrapolation.

The actor does not receive a cell-validity mask, shovel extension, the previous action, episode time remaining, particle positions, the simulator world pose, or any privileged current-minus-target map. Values outside the bounds above are contract failures, not values to clip.

The three actions are normalized to [-1, 1]: robot/cmd_vel on [0:2] (linear scale/saturation 0.4 m/s; angular scale radians(60) then saturation at 1.0 rad/s) and payload/joint_vel on [2:3] (scale/saturation 0.04 m/s).

Full coverage is a hard invariant

There is no validity mask, so every delivered frame must be complete. The source grid is fixed: 0.05 m cells, an 80 × 80 central target ROI, a 33-cell (1.65 m) guard on every side, and therefore a 146 × 146 complete source. A frame that is non-finite anywhere, incomplete, wrongly shaped, or outside the declared elevation/pose/velocity envelope is rejected whole — there is no sentinel fill, no silent clip, and no partial frame.

Unknown cells are never turned into plausible zero elevation. A cell with no supported particle receives the calibrated physical base-plane elevation, because the source is a semantic terrain-work-surface layer (containment base plane plus regolith) and excludes the rover, shovel, containment wall, and other transient occluders. If a real lab mapper cannot provide that coverage under the rover and shovel, the full-coverage decision must be reopened rather than approximated.

4. The external-heightmap source: timing, freshness, and the target manifest

The rover has no onboard depth camera. The actor sees only the product of one external mapper.

10 Hz contract. One accepted-or-held frame is published at every policy step.
Synchronized pose and velocity. The map, the mission pose, and the body velocity are captured in one provider snapshot and share one source timestamp. The local crop is sampled at that frame’s pose — an older map is never combined with a newer pose.
Source time. In simulation the declared clock is monotonic_sim_time (common_step_counter * step_dt), monotonic across episode resets. Producers reject future and non-monotonic timestamps.
Reset readiness. Reset blocks until the first complete frame exists, with a bounded 2.0 s timeout (20 polls at 10 Hz). Failure raises ExternalHeightmapReadinessError listing the failing environment ids. No zero startup frame is ever substituted.
Stale abort. The hard cutoff is 0.5 s, which is why the actor’s freshness bound is [0.0, 0.5] s. A frame older than that is rejected and handled as an out-of-MDP pause/abort — ExternalHeightmapStaleError in simulation, and a sensor_stale session abort on the deployment path.
Dropouts hold, they do not hole. A dropped update re-publishes the previous complete map and increases the freshness value.

Measurement profiles

Selected with env.external_heightmap.profile. Each profile is hashed (canonical JSON, SHA-256), and the hash travels in the run manifest:

Profile	Behaviour	Profile hash
`ideal` (default)	zero delay, dropout, elevation noise, registration error, pose jitter	`9c67bdce…2283398`
`randomized`	whole-frame delay `U[0.0, 0.1] s`; frame-drop probability `0.05` capped at two consecutive drops; per-cell elevation noise `N(0, 0.002 m)` clipped to `±0.006 m`; one per-episode SE(2) registration translation `N(0, 0.005 m)` clipped to `±0.015 m` and yaw `N(0, 0.25°)` clipped to `±0.75°`; per-frame pose jitter with the same bounds	`a008c11d…f94870fe`
`failure_stale`	at least six consecutive held updates — used only to prove the stale failure path	`192fbd06…3652a8e35`

These are seeded robustness defaults, not claims about lab error distributions and not a curriculum. Lab recordings may replace the randomized values only through a new versioned profile hash.

Measured agreement between the oracle (particle truth) and the observable (mapper product) evaluation, over 100 seeded frames on the crater grading cells: ideal agrees to 1e-9 with identical success classification; randomized has p95 disagreement 0.00189 m against a 0.01 m bound and 100 % success-classification agreement against a ≥ 95 % bound.

Only ideal has ever been exercised on a training run or on the general task.

The target manifest owns the mission identity

The manifest — not the code, not the config — owns the mission frame ID, the world-to-mission calibration version, the elevation datum, the source/ROI/guard geometry, the surface aggregation rule, the target map, the reset layout, the physical bounds, the grading-cell set, and the success parameters. Simulation and deployment both reject a mismatched frame, datum, geometry, or hash before policy inference.

Four hashes, never conflated. They are four different identities and conflating any two hides real drift:

Hash	What it identifies	Crater value
`io_schema_fingerprint`	the action/observation schema itself	`74bfe641…60ff16`
`target_map_sha256`	the desired elevation field	`f20f228f…f305bc8a`
`reset_layout_sha256`	the deterministic particle spawn layout	`6e809eb1…fd6769d4a`
`manifest_sha256`	the whole manifest document	`08e66df6…f1ec6f17`

The crater task’s manifest is checked in and fixed; configuration fails with an actionable error if any crater/pile/spawner field stops reproducing the hashed reset layout. The general task generates a manifest per episode from a deterministic (env.general_mission_seed, env_id, episode_index) stream — so a recorded triple regenerates the identical mission in any process — and its reset_layout_sha256 (b2b2cdbf…53a95b90) is fixed for the task while its target-map and manifest hashes vary per mission.

5. Particles: GPU solver, CPU-facing readback

Landscaping regolith is a PhysX PBD particle set, and the split between where it is simulated and where it is read matters operationally:

The solver is GPU-only. PhysX rejects particle sets outright when GPU dynamics are unavailable, logging Particles feature is only supported on GPU. Please enable GPU dynamics flag in Property/Scene of physics scene! and leaving every particle bit-identically inert. A CUDA-capable NVIDIA GPU is required.
The per-particle readback is the CPU-facing USD transport (UsdGeom.Points), fed by that GPU solver. On the pinned Isaac Sim 6.0.1 build there is no direct/Fabric per-particle alternative: omni.physics.tensors.SimulationView exposes only cloth, material and system-level particle views, and isaacsim.core.prims.ParticleSystem is system-level. This was probed and recorded, not assumed.
Both landscaping tasks therefore set sim.device = "cpu", and SRB disables Fabric automatically whenever particles are enabled. Isaac Sim 6’s CUDA direct-data pipeline does not synchronize PBD particle positions or velocities back to the USD points that the heightmap, the reward, and the renderer consume.

One trap worth knowing. A Kit-persisted app-global setting (/persistent/physics/overrideGPUSettings = 0, “Force CPU”) overrides the authored per-scene physxScene:enableGPUDynamics=true, so a machine that once had Force-CPU selected in the UI will silently produce a completely inert particle bed while everything else — rigid bodies included — behaves normally. This was the demonstrated cause of a long-standing zero-displacement symptom; neither the particle simulationOwner relationship nor USD readback synchronization had anything to do with it. SRB now clears that override at spawn time (with a warning) rather than trusting machine state, and authors an explicit simulationOwner that must equal the configured physics_prim_path.

Every landscaping particle claim is version-specific to Isaac Sim 6.0.1 / omni.physx 110.0.7 and must be re-proven after either changes, via tests/integration/test_particle_simulation_owner.py plus the two smoke nodes.

6. Dreamer

DreamerV3 is the only learner wired to the landscaping contract. It is pinned to 4049794d4135e41c691f18da38a9af7541b01553 with elements 3.22.0.

The task-level configuration lives at hyperparams/task/terrain_landscaping_crater/dreamerv3.yaml and is merged strictly: a dropped key is fatal, and the loaded config must declare every key the checked-in file declares. An operator profile that was never written against the strict upstream schema falls back to a lenient merge with a warning, and the run records strict_task_config so you can tell which happened.

num_envs > 1 is refused at wrapper construction, before any reset, step, or allocation, with an error naming the missing selective-reset transport and the qualified env.num_envs=1 path. See §7.

Checkpoint operations — three of them, named precisely

Operation	What it restores	What it does not restore
Same-logdir training-state resume (`--continue`)	`step`, the agent (parameters plus optimizer/update counters), and the replay buffer, in a fresh process	simulator/environment state, partial episodes, driver and RSSM carries, process RNG
Agent import / transfer	agent state only, into a new run with a new logdir, starting at step 0	everything else — it is never described as a resume
Evaluation / policy load	agent state only, for inference	everything else

The resume is not a bitwise continuation, and the exclusions above are the explicit contract, not an oversight. It requires synchronous, complete replay chunks: the run writes a ckpt_generation.json manifest recording the step, the replay item count, and the completed chunk paths, and a resume whose recorded chunks are missing or incomplete fails loudly rather than continuing on a silently empty replay. run.from_checkpoint is never used for this operation, and the process logs which operation it performed.

A portable model artifact for Dreamer is a sanitized directory, and it is not a run snapshot. It contains only the empty done marker plus exactly one agent payload form (agent.pkl xor a contiguous agent-NNNN.pkl shard set). Step and replay payloads, symlinks, nested directories and unrecognized members are structurally rejected without unpickling. Loading one starts a new run; it never resumes training.

Every Dreamer checkpoint and policy load runs a fail-closed contract preflight: the artifact’s recorded task id, IO-schema fingerprint, projection layout fingerprint, target-manifest hash, mapper-profile hash and normalization policy must all match the live environment, or the load is rejected before inference. The projection layout fingerprint is the order half — a checkpoint trained under a different packed-vector order is rejected even when its IO-schema fingerprint is identical. The only operator opt-out (allow_unverified=True / SRB_ALLOW_UNVERIFIED_LANDSCAPING_CHECKPOINT) tolerates an unverifiable artifact; it never tolerates an actual mismatch.

What Dreamer’s integration does and does not prove

G4 proves runtime viability of the one-environment strict path: the contract projection, strict configuration, upstream construct/update, truncation that resets collection while staying non-terminal for bootstrap, a 1,000-step smoke at ~14 env steps/s, checkpoint/reload/eval, the sanitized artifact path, and mismatch rejection.

It proves nothing about learnability, the general task, or hardware. Its former reliability caveat — the 1,000-step smoke aborted on the mission-pose envelope in 1 of 2 attempts of its 2026-08-01 re-measurement — is resolved (2026-08-07): on the corrected bed, five 1,000-step smokes (two ideal-profile, three randomized-profile) and six 10,000-step canaries all completed on the first attempt with zero envelope aborts.

It also does not establish that a long run fits. The G6 clause “projected replay/model memory fits the declared host envelope” is blocked on its own evidence, independently of G5: the 1,000-step smoke ends at 8.28 GB host RSS, but the peak over the 10,000-step canary campaign is 23.33 GiB — 97.2 % of the declared 24 GiB limit — at only ~3,300 steps. The limit was never breached and may not be raised after a failing run. The 2026-08-07 completed canaries re-evidence the clause (peaks at 66–73 % with a ~15.5 GiB plateau — see section 2), but nothing beyond 10,000 steps has been measured, so it stays blocked.

7. Batched training is unsupported

num_envs > 1 is refused fail-closed for Dreamer landscaping, and there are two independent measured reasons:

Mixed-reset transport. After one row truncates, the current transport advances the freshly reset row by one hidden physics step under a masked action before labelling that frame is_first=True. The continuing row is otherwise unaffected and no global reset occurs, but the reset row’s first observation is not its true unstepped reset observation.
Particle clone frames. ParticleSystem reads and writes per-particle state in prim-local coordinates while exposing it as world-frame buffers. With cloned environments carrying non-zero origin transforms, both beds collapse onto the world origin seam and both rover articulations read all-NaN. This is a core particle-transport defect, not a landscaping one.

Rejecting construction is the recorded outcome, deliberately, rather than weakening the mixed-reset requirement.

8. TD-MPC2 — landscaping is blocked

TD-MPC2 cannot be used for any landscaping task, and asking for it fails immediately with:

TD-MPC2 landscaping is disabled: velocity-controlled shovel extension is unobserved and no approved recurrent/history state contract exists.

The reason is observability, not plumbing. The shovel is velocity-controlled and its extension is not in the actor observation, so the same visible state can correspond to different shovel extensions. Upstream TD-MPC2 encodes the current state feed-forward and cannot recover that hidden actuator state (_prev_mean is planner state, not recurrent observation state). Dreamer is allowed a belief about extension only because its recurrent carry includes the previous action, and even that is an accepted partial-observability risk for a short canary rather than proof that extension is observed.

The block is capability/contract based, not name based — it fires on either frozen task ID in any spelling, on any environment publishing landscaping_contract_metadata(), and on any generated RealEnv declaring the contract’s IO-schema fingerprint. It fires before the log directory is created, before config.yaml is written, before checkpoint discovery, before replay allocation, and before the agent is constructed. Unblocking requires separate approval of one of: measured shovel extension, an approved observation/action history or recurrent encoder, or another physically grounded state estimator. Integrating commanded velocity and presenting the estimate as measured extension is explicitly forbidden.

What TD-MPC2 does work on

The generic dict-flattening path is repaired and usable on ordinary multi-leaf SRB tasks: every actor-visible numeric non-image leaf (packed vectors and 2-D map leaves) is concatenated into one float32 state vector with physical bounds, and image-like leaves are rejected at construction with an actionable message instead of reaching upstream’s encoder after the log directory, config, replay and model have already been created.

That path is gated on a pinned upstream checkout:

Item	Value
upstream base	`8bbc14ebabdb32ea7ada5c801dc525d0dc73bafe`
backport applied	`e9f59321933cbc8e11a002b842adc7d4ffae8ff1` (fix Q-ensemble weight init)
resulting pinned SHA	`bfb0029669f7242c33b1400950f97518ba46a5d8`
deliberately rejected	`75212c3a090115df212402ac911df446ccc2047f` — it pins Torch 2.7.1 / TensorDict 0.8.3 / TorchRL 0.8.1, older than Isaac Sim ships

The decision was to keep Isaac’s installed Torch/TensorDict/TorchRL matrix and backport only the Q-ensemble initialization fix. Without the backport the ensemble silently keeps PyTorch’s kaiming_uniform_ weights and non-zero biases instead of the intended trunc_normal_(std=0.02) with zeroed biases. assert_upstream_compatible() runs in both the train and policy-load paths before anything is allocated and refuses an unpatched checkout with the exact re-pin commands. The upstream contract test therefore checks the installed checkout, so its result is environment-dependent: it is green against the pinned patched checkout and red against an unpatched one, which is the gate working as designed.

TD-MPC2 checkpoints are weights-only

Upstream’s save() writes {"model": state_dict} and nothing else — no optimizer, replay, step, or RNG state. Therefore:

--model <ckpt> on train is a weights import. Training starts at an explicitly logged step 0 with a fresh replay buffer, fresh optimizers and reset schedules. The step is not parsed out of the file name.
--continue / continue_training=True fails immediately with a NotImplementedError explaining that no versioned full trainer-state checkpoint exists; nothing is allocated first.
an evaluation/policy load is logged as Evaluation load (weights only).

No TD-MPC2 weights-only load is a training resume, and no output describes one as such. Contrast this with Dreamer, which does have a real same-logdir training-state resume (§6).

9. Real Raph validation is unavailable

No hardware evidence exists. The hardware-free half of the sim-to-real work is complete — the observable task evaluator, the session-abort vocabulary, the horizon-truncation semantics, the model-artifact refusal, and a crater validation spec that resolves its real environment. The live half was never started: no live Raph or mapper contract has been recorded, and there is no lab run.

Constructing the canonical landscaping RealEnv validates four capability tags before it acquires a ROS node, starts hardware, or lets a caller load a policy:

Tag	Claimed by shipped code?
`landscaping.observable_task_evaluator`	yes — `LandscapingRealEvaluator`
`raph.drive_velocity`	no
`raph.shovel_prismatic_velocity`	no
`landscaping.external_heightmap_batch`	no

A CLI validation run of the crater spec therefore exits 4 with a typed DeploymentNotReadyError naming the missing tags. That is the intended and measured state.

What must be recorded from the lab before this can change: the Raph ROS drive/shovel command topics and message types, unit and sign conventions, saturation behaviour, watchdog/timeout semantics, acknowledgement mechanism, measured joint-state feedback, the measured real drive limits (the values in the contract are simulator values, not verified lab facts), the live external mapper and its calibration, live map/pose and map/velocity skew, and the live 10 Hz pacing and latency distribution.

Simulation and fake-adapter results are not hardware proof. Passing every other gate on this page would still not imply G8.

Known reporting hole. srb real_agent validate --dry-run skips environment and policy instantiation and finalizes with exit 0, pass_overall: true, n_episodes: 0, and a green real-eval: pass badge.json. Its three criteria are all reported with status skipped, which is the only in-band signal that nothing was validated. Do not treat a dry-run artifact as a validation result. Recorded, not fixed.

10. Generating and checking the deployment bridge

Both task IDs generate distinct bridge modules, and the general task must never be relabelled with the crater’s hash:

# Regenerate the checked-in bridge modules (writes in place)
srb real_agent gen --env terrain_landscaping_crater
srb real_agent gen --env terrain_landscaping

# Freshness gate: read-only byte comparison against the checked-in module
srb real_agent gen --env terrain_landscaping_crater --check
srb real_agent gen --env terrain_landscaping --check

Both run unqualified — no env.particles_height override belongs in them, and none must be reintroduced. --check renders and formats a candidate, byte-compares it against the checked-in module, prints a unified diff, and exits non-zero on drift without touching the file. It requires the repo’s pinned formatter (ruff) on PATH; without one it refuses to compare rather than manufacturing drift from an unformatted candidate. srb real_agent gen --env ALL --check runs the gate across every cached environment and exits non-zero if any drifted or if the cache is empty.

See srb real_agent and the Sim-to-Real workflow §5.2 for the generated schema and the deployment capability gate.

11. Configuration hazard worth knowing

srb.utils.hydra.extract.extract_defaults_from_class serializes an asset-instance field default as {"name": …} plus a small allow-list (action_mode and nested sub-asset fields). Any other customization the task declared on that instance is silently dropped, so a CLI/Hydra-launched run rebuilds the asset from its class defaults while a directly constructed TaskCfg(...) gets the declared values.

This was measured on landscaping: direct construction received a (4.0, 4.0) m containment with 0.3 m walls; the CLI path received Plane’s class defaults (0.75, 2.0) m with 1.0 m walls. Every CLI-launched landscaping run before the fix — crater included — trained on a surface a fraction of the declared size.

The landscaping tasks are repaired locally by declaring the containment as plain scalars (env.bed_size_m, env.bed_wall_height_m) that survive the round trip. The underlying allow-list is unchanged, so this remains a repo-wide hazard for any task that customizes an asset-instance default, and one cosmetic instance is still live in landscaping (the crater’s plane visual_material). If you customize an asset instance in a task config, verify it survives the Hydra round trip rather than assuming it does.

12. Evidence

Every claim on this page reduces one of the reviewed records in docs/superpowers/evidence/terrain_landscaping/:

Record	Subject
`A0.md`	frozen IO contract, target manifest, four hashes
`A1.md`	particle dynamics and readback, demonstrated cause
`A2.md`	cloned Raph locomotion, D4 fingerprint sync, G6-M blocker
`A3.md`	simple-task reward semantics, feasibility, G2 reachability campaign
`A4.md`	external-heightmap observation seam
`A5.md`	simulator / generated `RealEnv` / hardware schema alignment
`A6a.md`	Dreamer integration, strict config, checkpoint semantics (G4)
`A6b.md`	bounded canaries and the G5 model gate (blocked)
`A7.md`	TD-MPC2 generic repair, upstream pin, landscaping preflight
`A8a.md`	truthful real validation, hardware-free half
`A10.md`	general task repair, feasibility, G9 reachability
`A9.md`	this reduction
`2026-08-07-recheck.md`	training-path recheck, the corrected-bed re-measure
`2026-08-07-mound-spike.md`	mound-geometry feasibility spike, volume-metric separation
`2026-08-07-mound-margin-derivation.md`	the mound volume margin and the paired protocol
`2026-08-08-mound-ladder.md`	the mound-scoped ladder and its model-gate failure
`readiness.json`	the machine-readable gate report

A8b.md does not exist: the live hardware half was never started.

Environment Configuration

The Space Robotics Bench provides a flexible configuration system for environments through Hydra. This document explains how to customize environment parameters across different domains and tasks.

How Does It Work?

Each SRB environment is registered alongside its Python configuration class, which defines the default parameters for that specific environment. All environment configuration classes are organized in a hierarchical structure via inheritance, where BaseEnvCfg does most of the heavy lifting. This design supports a modular and extensible configuration system that allows for easy customization of environment parameters.

Modifying Configurations

You can modify environment configurations in several ways:

1. Command-Line Overrides

The most direct way to modify environment parameters is through command-line overrides:

srb agent <WORKFLOW> --env <ENV> \
    env.domain=moon \
    env.robot=ur10 \
    env.num_envs=4 \
    env.stack=false \
    ...

2. Configuration Files

Each SRB environment is backed by a Python configuration class (a @configclass-decorated dataclass extending BaseEnvCfg). When you launch a task, Hydra extracts the default values from this class and merges them with any command-line overrides.

To inspect the full set of defaults for any environment, read its Python configuration class directly (shown below) — Hydra extracts these defaults from the class at launch. Every run also dumps its resolved configuration to <logdir>/.hydra/config.yaml, which doubles as a reference for the expected YAML structure.

If you want to persist a set of overrides, create a YAML file and pass its path via the --cfg flag:

srb agent zero --env peg_in_hole --cfg ./my_overrides.yaml

where my_overrides.yaml contains:

env:
  domain: mars
  num_envs: 8
  robot: ur10

Note: --cfg also accepts default (use the environment’s default config) and ignore/none/null (no config file).

Tip: Shortcut syntax is supported for assets — env.robot=franka is automatically expanded to env.robot.name=franka.

Key Parameters

Below are the most important configuration parameters organized by category (several parameters have their own detailed documentation pages):

Scenario/Environment

env.domain - Domain
env.robot - Robot
env.num_envs, env.stack - Parallelism
env.particles - Particles
env.reward - Reward & Normalization

Simulation

env.sim - Low-level simulation parameters (physics, rendering, etc.)
env.visuals - Visual appearance settings

Debugging

env.debug_vis - Enables debug visualization features

Environment Configuration — Parallelism

One of Space Robotics Bench’s most powerful features is parallel simulation, allowing you to run multiple simulation instances simultaneously. This capability is critical for:

Reinforcement Learning: Collect experience at scale for faster training
Parameter Tuning: Test multiple configurations simultaneously
Monte Carlo Sampling: Evaluate robustness across varied scenarios
Batch Processing: Process multiple scenarios in a single run

Number of Environments

The env.scene.num_envs parameter controls how many parallel simulation instances are created. This parameter is aliased as env.num_envs for brevity:

srb agent rand -e _manipulation env.scene.num_envs=16
srb agent zero -e _manipulation env.num_envs=128

Each environment is a fully independent physics simulation instance, with all cross-environment interactions disabled (filtered collisions). However, there is only one instance of the rendering engine, which means that a visual sensor in one environment will see entities from all environments. A simple workaround for this limitation is to increase the spacing between environments and clip the maximum sensor range.

Environment Spacing

The env.scene.env_spacing parameter controls the spatial distance between environments. This parameter is aliased as env.spacing for brevity:

srb agent rand -e _manipulation env.num_envs=64 env.scene.env_spacing=20.0
srb agent rand -e _manipulation env.num_envs=32 env.spacing=10.0

Note: When env.stack=true, the framework forces env.scene.env_spacing=0.0 regardless of the value of env.spacing, since stacked environments share the same scenery position.

Environment Stacking

The env.stack parameter controls whether environments share assets or have independent assets:

Independent Environments (`env.stack=false`)

⚠️ Each environment has a unique scene and position
✅ Greater visual and physical diversity
✅ Supports tasks with visual sensors
❗ Slower to initialize
❗ Higher memory usage

srb agent rand -e _manipulation env.num_envs=16 env.stack=false

Stacked Environments (`env.stack=true`)

⚠️ All environments share the same scenery and position
✅ Faster to initialize
✅ Lower memory usage
❗ Less environmental diversity
❗ Does not support tasks with visual sensors
❗ Does not support particle or cloth simulation with multiple environments — when env.particles=true or env.cloth=true and env.num_envs != 1, the framework forces env.stack=false (a warning is logged)

srb agent rand -e _manipulation env.num_envs=16 env.stack=true

Note: The default is task-specific. Non-visual locomotion_velocity_tracking and the single-env public excavation_demo default to env.stack=true.

Environment Configuration — Domain

The env.domain parameter configures the planetary or orbital environment for your simulation. This affects gravity, lighting conditions, terrain type, atmospheric properties, and visual appearance.

Available Domains

Space Robotics Bench currently supports the following domains:

Domain	Description
`moon` [default]	Lunar surface
`mars`	Martian surface
`earth`	Earth surface
`asteroid`	Low-gravity body with irregular terrain
`orbit`	Orbital microgravity environment

Usage

You can specify the domain via the command line:

# Run a simulation on Mars
srb agent teleop --env sample_collection env.domain=mars

Domain-Specific Effects

Each domain affects various aspects of the simulation:

Physical Properties

Gravity magnitude: Affects object dynamics, robot mobility and manipulation requirements

Visual Properties

Lighting: Light intensity, color temperature, and angular diameter
Skydome: Background appearance and intensity
Atmosphere: Modeled through the domain’s light intensity, color temperature, and Sun angular diameter (fog itself is a separate manual env.visuals.fog setting, independent of the domain)

Asset Selection

Some assets are domain-specific and will only appear for compatible domains. When an asset is registered, it can specify the domains it supports through the DOMAINS class attribute.

Environment Configuration — Robot

The env.robot parameter allows you to specify which robot to use in your simulation. This is one of the most powerful configuration options, enabling you to change not just the robot model but also its components, payloads, and end effectors.

Tip: Run srb ls to see every registered robot (and which category it belongs to). The -e value in the examples below selects an environment or environment category — categories are prefixed with an underscore, e.g. _manipulation, _ground, _aerial. A robot is only valid for environments whose category it supports.

Robot Selection

The simplest way to select a robot is by specifying its name:

srb agent rand -e _manipulation env.robot=ur10

srb agent rand -e _ground env.robot=unitree_go1

srb agent rand -e _aerial env.robot=crazyflie

End Effector for Manipulators

You can attach a specific end effector to a manipulator with the base+end_effector syntax. Both the base manipulator and the end effector must be named:

srb agent rand -e _manipulation env.robot=ur10+shadow_hand

Payload for Mobile Robots

Similarly, you can attach a payload to a mobile robot with the base+payload syntax:

srb agent rand -e _ground env.robot=spot+cargo_bay

Payloads may be static rigid geometry or an ActivePayload articulation. An active payload contributes its own action terms after the mobile base terms and is mounted as a separately addressable scene articulation. For example, the landscaping configuration below has three actions: two filtered RaphRover drive channels followed by one continuous RaphShovel velocity channel.

srb agent rand -e terrain_landscaping env.robot=raph_rover+raph_shovel

That landscaping action vector is frozen by the task’s IO contract: three normalized channels in [-1, 1], mapped to robot/cmd_vel on slice [0:2] (linear scale and saturation 0.4 m/s; angular scale radians(60) followed by saturation at 1.0 rad/s) and payload/joint_vel on slice [2:3] (scale and saturation 0.04 m/s). Those are simulator limits, not verified lab facts; see Terrain Landscaping Training Readiness.

Hazard — asset-instance customizations can be dropped on the CLI path. When a task config is rebuilt through Hydra (which is what every srb agent / srb real_agent invocation does), extract_defaults_from_class serializes an asset-instance field default as {"name": …} plus a small allow-list (action_mode and nested sub-asset fields). Anything else the task declared on that instance is silently dropped, and the asset is rebuilt from its class defaults. A directly constructed TaskCfg(...) keeps the declared values, so the two paths can disagree without any error. This was measured on the landscaping containment: direct construction received a (4.0, 4.0) m surface with 0.3 m walls, the CLI path received Plane’s class defaults (0.75, 2.0) m with 1.0 m walls. If you customize an asset instance in a task config, express it as plain scalars re-applied in __post_init__ (what the landscaping tasks now do), or verify explicitly that it survives the round trip.

Combine Mobile Manipulators

SRB supports combining any mobile robot with any manipulator for mobile manipulation environments via the env.robot.mobile_base and env.robot.manipulator parameters:

srb agent rand -e _ground_manipulation env.robot.mobile_base=unitree_a1 env.robot.manipulator=unitree_z1

Similarly, you can attach payloads to the mobile base and end effectors to the manipulator:

srb agent rand -e _ground_manipulation env.robot.mobile_base=anymal_d+cargo_bay env.robot.manipulator=ur10+shadow_hand

Alternatively, a ground mobile manipulator can be composed in a single env.robot token — the first name must be a registered mobile robot and the second a registered manipulator, which may itself carry an end-effector suffix:

srb agent rand -e _ground_manipulation env.robot=unitree_a1+unitree_z1
srb agent rand -e _ground_manipulation env.robot=husky+kinova_gen3n7+robotiq_hand_e

Change Action Modes

Each robot, active payload, and active tool (end-effector) contributes its selected action mode to the action space of the whole system. You can change the selected mode with action_mode on the robot, mobile base, manipulator, or active tool:

srb agent rand -e _manipulation env.robot.action_mode=joint_pos_rel

Robot selection and action mode overrides compose:

srb agent rand -e _manipulation env.robot=ur10 env.robot.action_mode=joint_pos_rel

Environment Configuration — Particles

Space Robotics Bench can simulate liquid and granular materials like regolith, sand, and dust using particle-based physics. This is particularly relevant for space applications where interaction with loose granular material is common.

Enabling Particles

You can enable particles in any environment by setting the particles parameter to true:

srb agent teleop -e _manipulation env.particles=true env.robot=franka+scoop

Particle Configuration Parameters

Most particle parameters accept either a single scalar (used for every particle system) or a (min, max) 2-tuple that is interpolated across env.particles_systems to produce a graded distribution. Defaults below are taken from BaseEnvCfg (srb/core/env/common/base/env_cfg.py).

Parameter	Description	Default
`env.particles`	Enable particle simulation	`false`
`env.particles_systems`	Number of independent particle systems per environment	`1`
`env.particles_spawner`	Pile spawner kernel — `grid` or `pyramid`	`grid`
`env.particles_size`	Particle diameter in metres (scalar or `(min, max)` 2-tuple)	`(0.01, 0.005)`
`env.particles_area`	Footprint of the pile (scalar or 2-tuple)	`(0.75, 0.5)`
`env.particles_ratio`	Volume-fill ratio (scalar or 2-tuple)	`(0.025, 0.1)`
`env.particles_height`	Stack height per system (integer scalar or 2-tuple)	`(1, 10)`
`env.particles_density`	Bulk material density (kg/m³)	`1600.0`
`env.particles_friction`	Particle–particle friction coefficient	`1.0`
`env.particles_damping`	Velocity damping	`0.2`
`env.particles_cohesion`	Cohesion (particle–particle stickiness)	`0.01`
`env.particles_adhesion`	Adhesion (particle–surface stickiness)	`0.05`
`env.particles_material`	Visual-material override for the particle set (advanced — a `VisualMaterialCfg`, not a scalar)	`None`
`env.particles_dim_xy`	Override grid count `(dim_x = dim_y)` per system	`None` (derived)
`env.particles_init_pos`	Explicit `(x, y, z)` placement of the first system	`None` (derived)
`env.particles_freeze`	Freeze systems after they settle — the solver stops but the settled geometry keeps rendering (for clean renders)	`false`
`env.particles_conform_to_terrain`	Snap particles onto the terrain height field instead of PBD settling	`false`
`env.particles_conform_jitter`	Random XY scatter (metres) applied when conforming to terrain (`0` keeps the regular grid)	`0.0`
`env.particles_pile_center_jitter`	Per-reset random XY offset (metres) of the whole settled pile — domain randomization of pile location (`0` disables)	`0.0`

srb agent teleop -e _manipulation env.particles=true \
  env.particles_size=0.01 env.particles_ratio=0.1

Particle Behavior

The default grid spawner produces a regular tessellation of particles within particles_area. Switch to particles_spawner=pyramid for natural-looking piles with higher density at the centre. Multi-system stacking (particles_systems>1) layers piles vertically and supports gradients across particle size, area, ratio, etc. Particles collide with rigid bodies and articulations, settle under gravity, and can be pushed, scooped, or otherwise manipulated by the agent.

In the particle tasks (excavation, terrain_landscaping) the agent additionally observes a heightmap of the particle surface. PhysX particle sets are not raycastable, so this heightmap is derived at runtime from the live particle positions rather than from a ray sensor — it tracks the pile as it is reshaped. The landscaping tasks route that derivation through a versioned external-heightmap seam with a fixed footprint-aware surface rule, not through ad-hoc binning.

Note: When particles are enabled, Fabric is automatically disabled (env.sim.use_fabric=false) so that the underlying PhysX particle simulation receives consistent state.

Where Particles Are Simulated vs. Where They Are Read

These are two different places, and the distinction is operationally load-bearing:

The solver is GPU-only. PhysX PBD particles require GPU dynamics. When GPU dynamics are unavailable, PhysX rejects every particle set at parse time with Particles feature is only supported on GPU. Please enable GPU dynamics flag in Property/Scene of physics scene! and the particles stay bit-identically inert — while rigid bodies in the same stage keep behaving normally. A CUDA-capable NVIDIA GPU is required for any particle task.
The per-particle readback is the CPU-facing USD transport (UsdGeom.Points), fed by that GPU solver. On the pinned Isaac Sim 6.0.1 build there is no direct/Fabric per-particle alternative: omni.physics.tensors.SimulationView exposes only cloth, material and system-level particle views, and isaacsim.core.prims.ParticleSystem is system-level. This was probed on the installed build, not assumed.
Consequently the landscaping tasks pin sim.device = "cpu". Isaac Sim 6’s CUDA direct-data pipeline does not synchronize PBD particle positions or velocities back to the USD points consumed by the heightmap, the reward, and the renderer.

Trap — a persisted Kit setting can silently disable all particles. The app-global /persistent/physics/overrideGPUSettings value overrides the authored per-scene physxScene:enableGPUDynamics=true. A machine on which “Force CPU” was ever selected in the UI keeps 0 in kit/data/Kit/IsaacLab/*/user.config.json and will produce a completely inert particle bed in every scene. SRB now clears that override at spawn time (with a warning) instead of trusting machine state. Measured enum: -1 = use per-scene settings, 0 = force CPU, 1/2 = force GPU.

Each spawned particle set also authors an explicit simulationOwner relationship, which must equal the configured env.sim.physics_prim_path; a missing or non-UsdPhysics.Scene owner fails fast rather than falling back to stage traversal. Note that ownership is a determinism invariant, not a motion prerequisite — an unowned set still moves, and blaming a missing owner for inert particles is a diagnosis error the repository has already made once.

Known limitation — cloned environments. Per-particle state is read and written in prim-local coordinates while being exposed as world-frame buffers, so with env.num_envs > 1 the beds of cloned environments collapse onto the world origin seam. Particle tasks are currently single-environment only.

Domain Randomization

Regolith interaction is a prime sim-to-real gap, so a few knobs randomize the granular material to keep a policy from overfitting one fixed pile:

env.particles_pile_center_jitter — on every reset the settled pile is restored and rigidly translated in XY by an independent per-environment sample in [-jitter, +jitter]. The pile keeps its settled shape (the translation is physically valid without re-settling), so 0.05 scatters where the pile sits by ±5 cm without changing its profile. 0 (the default) keeps the pile fixed.
The excavation and terrain-landscaping tasks additionally randomize the PhysX PBD particle material on every reset — friction is sampled in [0.6, 1.0] and density in [2700, 3100] kg/m³ — so the settled angle of repose and bulk behaviour vary trial-to-trial. This is wired into the tasks themselves rather than exposed as an env.* knob; in those two tasks env.particles_friction / env.particles_density therefore set the spawn baseline, not a per-episode constant.

Static Renders

For clean stills or offline datasets you often want the pile visible but motionless:

env.particles_freeze — after the initial settle completes, each system is disabled (its PhysX particleSystemEnabled flag is cleared and velocities zeroed). The settled points keep rendering, but the solver no longer simulates them, so the particles stay perfectly still. Pair it with the --snapshot capture for a frozen render bed.
env.particles_conform_to_terrain (optionally with env.particles_conform_jitter) snaps the spawned grid onto the terrain height field instead of PBD settling — useful when PBD settling is unavailable and you just need regolith that follows the terrain relief.

Reward & Normalization Configuration

In reinforcement learning, the design and shaping of the reward function are critical to policy convergence and stability. The Space Robotics Bench (SRB) provides a modular, declarative, and robust reward and normalization configuration system built on Pydantic models.

This guide explains how rewards are defined, the available reward term types, and how auto-normalization scales returns across diverse task horizons.

1. Defining a Reward Configuration

Each task exposes a specific RewardCfg class inheriting from BaseRewardCfg (or domain-specific subclasses like GroundMobileRewardCfg or LocomotionRewardCfg).

You usually don’t need to edit Python. To retune an existing reward term, pass a CLI override — e.g. env.reward.penalty_action_rate.weight=-0.2. To inspect a task’s default reward terms, read its RewardCfg class in source. The class below shows how those defaults are declared.

Here is an example structure:

from srb.core.reward import BaseRewardCfg, LinearRewardTerm, GaussianRewardTerm

class RewardCfg(BaseRewardCfg):
    # Smoothness & efficiency penalties
    penalty_action_rate: LinearRewardTerm = LinearRewardTerm(weight=-0.5, limit=4.0)

    # Task specific target rewards
    reward_target_approach: GaussianRewardTerm = GaussianRewardTerm(
        weight=4.0, scale=0.5
    )

At every environment step:

The environment’s extract_step_return() computes step metrics (e.g., current action variance or distance to target).
The metrics are passed to their corresponding reward terms inside the task dictionary.
SRB automatically aggregates all non-ignored RewardTerm instances, sums their computed values, and applies optional normalization before returning the final reward to the RL agent.

2. Types of Reward Terms

Most reward term classes are defined in srb.core.reward.terms (a few, such as TukeyRewardTerm, live in srb.core.reward.normalize); all are exported from srb.core.reward. They generally inherit from RewardTerm and are designed to shape physical metrics mathematically.

📐 Linear and Clamped Terms

LinearRewardTerm: Computes a linear function of the absolute value of the input metric, clamped by a specified maximum/minimum reward limit. (For a sign-preserving variant, use ScaledRewardTerm.) $$\text{Reward} = \text{clamp}(\text{weight} \times |\text{metric}|, -\text{limit}, \text{limit})$$
QuadraticRewardTerm: Applies a quadratic scaling to penalize larger errors exponentially.
OffsetLinearRewardTerm / OffsetQuadraticRewardTerm: Introduces a deadband threshold. No penalty is applied until the metric exceeds the specified threshold.

🎯 Kernel and Tracking Terms

GaussianRewardTerm: A radial basis function that provides a smooth bell-shaped curve. Highly effective for rewarding precise positioning. $$\text{Reward} = \text{weight} \times \exp\left(-\frac{\text{metric}^2}{2 \, \text{scale}^2}\right)$$
GaussianTrackingRewardTerm: Extends the Gaussian reward to specifically track a dynamic target value, matching a given command.
GaussianZoneTrackingRewardTerm: Tracking reward with a flat tolerance band (deadband zone) around the target value.

🛡️ Barrier Terms

HyperbolicBarrierRewardTerm / GaussianBarrierRewardTerm: Triggers a steep asymptotic penalty when a metric crosses a safety threshold (e.g., ground clearance or joint limit violations).

3. Reward Normalization

SRB features a robust normalization system to prevent specific tasks or custom configurations from dominating the reward budget. This ensures stable learning across very different environments.

Under the hood, BaseRewardCfg automatically compiles the theoretical worst and best step returns by summing the signed reward_bounds (worst, best) extremes of all registered reward terms. For one-sided terms this reduces to bucketing each limit by the sign of its weight; sign-preserving terms (e.g. ScaledRewardTerm, SmoothMaxRewardTerm) contribute to both sides:

$$R_{\text{worst}} = \sum_{T} \text{worst}(T) \qquad R_{\text{best}} = \sum_{T} \text{best}(T)$$

Normalization Modes

You can control how scaling behaves via the normalize parameter on the reward configuration:

Normalization Mode	Value	Behavior & Mathematics	Recommended For
`NONE`	`"none"`	sum is returned exactly as-is.	Custom manual debugging or classic benchmarks.
`PER_STEP`	`"per_step"`	Divided by the maximum step scale: $S = \max(R_{\text{best}},	R_{\text{worst}}
`EPISODE`	`"episode"`	Divided by the full episode scale: $S = \max(R_{\text{best}},	R_{\text{worst}}
`AUTO` (Default)	`"auto"`	Automatically resolves to `EPISODE` for finite-horizon tasks, and `PER_STEP` otherwise.	Standard default for all environments.

4. Normalization Troubleshooting & Best Practices

Tip

Always verify that your reward terms are balanced! If one term has a extremely large limit (e.g., limit=1000.0), it will distort the scale factor, rendering other smaller terms completely negligible under normalization.

To inspect and validate your task’s active bounds and normalized scale, you can initialize your task and print the bound introspection metrics:

reward_cfg = YourTaskRewardCfg()
worst, best = reward_cfg.get_bounds()
print(f"Worst per-step: {worst}, Best per-step: {best}")

This bounds check helps ensure that no single term dominates and that the normalization range is tight and uniform.

Environment Configuration — Ephemeris

The srb.core.ephemeris subsystem computes real sun/planet geometry — azimuth, elevation, distance, angular diameter, and solar-irradiance scale — for a given body-surface site and UTC epoch, using NASA/JPL’s NAIF SPICE toolkit via the spiceypy bindings. It replaces the hand-authored elevation/azimuth presets that ship with SRB’s lighting catalogs when you need geographically and temporally accurate lighting — for example, matching a specific Apollo 17 EVA timestamp or a Mars-rover landing epoch, or auditing whether a lunar polar site experiences a grazing sunrise on a given day.

The core query API and srb ephemeris CLI (Phase 1), ephemeris-driven terrain lighting (Phase 2 — a real-terrain site can bake its Sun/Earth lights from ephemeris data at a curated epoch; see Terrain lighting below), ephemeris-driven environment lighting (Phase 3 — an opt-in env.ephemeris config drives a running environment’s Sun light directly from ephemeris data, with optional per-reset epoch randomization; see Environment Wiring below), and ephemeris-driven orbit & dynamics (Phase 4 — a Domain.ORBIT circular-orbit observer, analytic eclipse, a moving sun, phase-dependent earthshine, an opt-in observation vector, and deterministic skydome co-drive; see Environment Wiring (Phase 4) below), and precomputed illumination products (Phase 5 — a kernel-free "table" provider drives the same env path from imported lunarlab/PGDA illumination rasters instead of live SPICE queries, plus PSR-aware spawn/reward consumers and epoch-window curricula; see Illumination Products (Phase 5) below) are all available today.

Installation

The core srb install does not pull in spiceypy — it is an opt-in extra so that users who don’t need real ephemeris data avoid the ~30 MB wheel:

pip install 'srb[spice]'

srb[spice] is also included in the srb[all] umbrella extra. Calling into the SPICE backend without the extra installed raises srb.utils.extras.MissingExtraError with the exact pip install hint rather than an opaque ModuleNotFoundError.

Kernel Bundle

Real ephemeris queries require a set of NAIF SPICE kernels — binary/text data files describing body positions (.bsp), orientation (.bpc/.tf), leap seconds (.tls), and physical constants (.tpc). SRB pins a single ~43 MB bundle (order matches furnish order) that covers the Sun, Earth, Moon, and Mars from 1849-12-26 to 2150-01-22:

File	Purpose	SHA-256
`naif0012.tls`	Leap-second kernel (current through the last leap second, 2016-12-31)	`678e32bdb5a744117a467cd9601cd6b373f0e9bc9bbde1371d5eee39600a039b`
`pck00011.tpc`	Planetary constants (radii, orientation models, `IAU_MARS`/`IAU_EARTH` frames)	`3dff7b1dbeceaa01f25467767d3fa25816051c85d162d1edf04acb310ee28bb1`
`de440s.bsp`	Planetary/lunar ephemeris (positions of Sun, Earth, Moon, Mars barycenter), 1849–2150	`c1c7feeab882263fc493a9d5a5b2ddd71b54826cdf65d8d17a76126b260a49f2`
`moon_pa_de440_200625.bpc`	Lunar orientation (principal-axis, `MOON_ME` frame)	`60cd55aa401ea2ea97360636f567554bfe4e37bb829f901b4460a455dfaf783f`
`moon_de440_250416.tf`	Lunar frame kernel (`MOON_ME` alias definitions)	`a47c71e9c9f33796bdafb2c9d69a7ee447b6016ecad80f71cd6f3e479f9cf768`

The pinned filenames, URLs, and hashes live in srb/core/ephemeris/kernels.py’s KERNEL_BUNDLE — treat that module as the single source of truth; this table is copied from it and must be kept in sync.

Fetching kernels

Kernels are fetched lazily on first use (SHA-256 verified, atomic download — the same srb.utils.artifact_fetch machinery used for event-camera and terrain assets) unless auto-fetch is disabled. To pre-fetch explicitly:

srb ephemeris download

Attribution and redistribution

Kernels originate from NAIF’s public archive: https://naif.jpl.nasa.gov/pub/naif/generic_kernels. NAIF’s rules page explicitly permits redistribution of unmodified kernels, which is what enables SRB’s primary mirror (a Hugging Face dataset repo, configured via SRB_EPHEMERIS_HF_REPO_ID) with the original NAIF URL always kept as a fallback so kernel access never depends on a single host. If you use ephemeris-derived data in published work, acknowledge SPICE/NAIF (Acton 1996, “Ancillary data services of NASA’s Navigation and Ancillary Information Facility”; Acton et al. 2018).

Environment Variables

Variable	Default	Purpose
`SRB_EPHEMERIS_CACHE_ROOT`	`<SRB repo>/.cache/ephemeris`	Directory kernels are downloaded into and read from. Override to share a kernel cache across checkouts or point at a pre-provisioned, read-only mount.
`SRB_EPHEMERIS_AUTO_FETCH`	`1` (enabled)	Set to `0`/`false` to disable on-demand downloading — useful for air-gapped CI. When disabled and kernels are missing, `resolve_kernels()` raises `KernelsMissingError` naming the exact `srb ephemeris download` command to run instead of silently reaching for the network.
`SRB_EPHEMERIS_HF_REPO_ID`	unset	Hugging Face dataset repo id (`org/name`) used as the primary kernel mirror, ahead of the NAIF fallback URL. Unset means requests go straight to NAIF.

CLI Usage

See the full verb reference at srb ephemeris. Two quick examples:

# Pre-fetch the pinned kernel bundle into the cache
srb ephemeris download

# Query sun/earth geometry for a Shackleton-rim-like site
srb ephemeris info --body moon --lat -89.66 --lon 0.0 \
  --utc 2026-11-01T12:00:00 --bodies sun,earth

Programmatic Usage

from srb.core.ephemeris import SiteSpec, query

site = SiteSpec(body="moon", lat_deg=-89.66, lon_deg=0.0, alt_m=0.0)
state = query(site, "2026-11-01T12:00:00", bodies=("sun", "earth"))

sun = state.bodies["sun"]
print(f"az={sun.azimuth_deg:.2f} el={sun.elevation_deg:.2f}")
print(f"irradiance_scale={state.irradiance_scale:.4f}")  # (1 AU / distance)^2

query() dispatches to a registered EphemerisProvider. The "spice" backend (srb.core.ephemeris.spice_provider) performs live kernel queries. The "table" backend (srb.core.ephemeris.table_provider) consumes imported srb_illum/2 products without a spiceypy dependency at query time. Terrain lighting bakes still use the SPICE-oriented bridge; table products are not a drop-in terrain-lighting backend.

Terrain lighting

A real-terrain site (see Real Terrain Assets) can drive its Sun (and, on the Moon, Earthshine) lights from real ephemeris data instead of a hand-authored static preset. Set the site’s lighting companion to the ephemeris sentinel in the terrain manifest (srb/terrain/manifest.yaml); the site must also declare its surface coords and a curated epoch (UTC):

shackleton_rim:
  body: moon
  coords: { lat: -89.66, lon: 0.0 }
  lighting: ephemeris          # SPICE-driven lighting at the site epoch
  epoch: "2026-11-01T12:00:00" # required when lighting == ephemeris
  # ...

At bake time the bridge (srb/terrain/lights/ephemeris_bridge.py) runs a single query() for the site/epoch, maps the result to a LightingPreset (Sun intensity scaled by solar irradiance and a night-policy visibility ramp; color temperature and base intensity inherited from the body-default preset), and writes an epoch-keyed lights.usd into a content-hashed cache slot under .../srb_lights/ephemeris/<body>_<site>_<epoch>_<hash8>/. A changed epoch or grid-north correction yields a fresh file; identical inputs reuse one.

Grid-north convergence (γ). The Sun’s true-north azimuth from query() is folded onto the terrain’s grid north via az_grid = az_true + γ, where γ (north_convergence_deg) is read from the baked terrain’s persisted georef block (its sibling meta.json) — never recomputed. γ is 0 for equirectangular / lon-0 sites and non-zero for polar-stereographic patches.

Provenance nuance. srb asset list terrain prints the site’s Sun geometry as the raw true-north azimuth straight from the ephemeris (sun=EL°@AZ° (ephemeris @ <utc>, true-north)), whereas the baked lights.usd folds in +γ. The two match only where γ = 0 (e.g. shackleton_rim at lon 0); for a polar-stereographic site expect the baked cast-shadow bearing to differ from the listed azimuth by γ.

Offline fallback. Ephemeris lighting needs the pinned NAIF kernels on disk. When they are absent the bridge does not query or download: with the default on_missing="fallback" it emits a single warning per (site, body, epoch) (warn_ephemeris_lighting_fallback_once) and returns the body-default static preset (lunar_default / mars_default), so an air-gapped run still bakes and renders; on_missing="error" raises KernelsMissingError instead. Run srb ephemeris download to enable the real path. srb asset list terrain shows (ephemeris @ <utc>, kernels missing) for such a site without attempting a query.

Flipping a site’s lighting from ephemeris back to a named static preset is config-only and produces byte-identical output to the pre-ephemeris bake — the feature is inert when unused.

Environment Wiring (Phase 3)

BaseEnvCfg.ephemeris: EphemerisCfg (srb/core/ephemeris/config.py, default EphemerisCfg(), i.e. enabled=False) drives a running environment’s shared Sun light (/World/sunlight) directly from real SPICE geometry — no terrain bake required. The field is never None — only its enabled flag gates the behavior — so the default (disabled) path never calls query(), never touches the scene graph, and is byte-identical to dev:

# Fixed epoch: query once, at config/scene-build time
srb agent teleop -e _ground \
  env.ephemeris.enabled=true \
  env.ephemeris.epoch.utc=2026-11-01T12:00:00

# Sampled epoch: draw a fresh epoch (and therefore sun position) every reset,
# seeded from env.seed for reproducibility
srb agent zero -e waypoint_navigation \
  env.ephemeris.enabled=true \
  env.ephemeris.epoch.utc_range='[2026-01-01T00:00:00,2026-12-31T00:00:00]'

EpochSpec takes exactly one of utc (fixed) or utc_range (sampled):

Fixed (epoch.utc). _add_sunlight_from_ephemeris queries once, at config time, and builds /World/sunlight’s orientation (via light_quat_xyzw) and radiometry from the result. A fixed epoch that puts the sun at or below the horizon is valid — it is simply night at that site/epoch — and only warns once rather than failing.
Sampled (epoch.utc_range). The config-time sun is a static placeholder; a mode="reset" event term (randomize_sun_ephemeris, srb/core/mdp/events.py) re-queries and re-applies the sun on every reset, drawing the epoch uniformly from the range via a dedicated np.random.default_rng(env_cfg.seed) stashed on env.unwrapped — two envs built with the same seed draw the same epoch sequence. A wide range can silently produce black episodes. sun_light_params zeroes intensity outright once the sun is at or below the horizon (sun_visibility_factor returns exactly 0.0 below -angular_radius), so a utc_range spanning a full day/night cycle at the resolved site — e.g. the year-long example above, which covers many lunar synodic months (~29.5 Earth days each) — yields roughly half fully dark episodes on the Moon. This is deliberate domain randomization, not a bug, but for a vision-based task it can be training-destroying if unnoticed: narrow utc_range if dark episodes are unwanted, or budget for them in the task/reward design. The first dark reset for a given site logs a one-time warning (mirroring the fixed-epoch below-horizon warning above).

Either way, /World/sunlight is a single global prim (there is no per-env sun), and all four static sun randomizers (randomize_sunlight_orientation, _intensity, _angular_diameter, _color_temperature) are disabled automatically — leaving even one enabled would let it clobber the ephemeris-derived orientation/radiometry on its next interval tick.

drive_sunlight (default true) gates whether the ephemeris drives /World/sunlight at all. drive_skydome (default false) is honored as of Phase 4 — see Skydome co-drive below: when both drive_sunlight and drive_skydome are true, the skydome’s world yaw is co-driven deterministically from the same sun geometry, and setting drive_skydome=true without drive_sunlight=true is a config error (check_ephemeris_precedence raises) rather than a silently inert or warned-about combination.

Site resolution. env.ephemeris.site (SiteSpec | None, default None) picks the observer location. Precedence, resolved once per env build:

An explicit env.ephemeris.site — always wins (and its body must match env.domain’s body, or building the env raises).
A real-terrain scenery: if env.scenery resolves to a RealTerrain whose curated manifest entry declares a site and whose body matches env.domain, that site’s coords become the observer location, and the terrain’s baked grid-north convergence γ (persisted in its georef metadata) is folded in as SiteSpec.north_yaw_deg = −γ. This looks like the opposite sign from Phase 2’s terrain-lighting az_grid = az_true + γ convention, but the two are equivalent (an active/passive rotation duality) — verified both analytically and numerically. Treat the sign as settled; do not re-derive it. A terrain whose body disagrees with env.domain is ignored (with a warning), never silently relabeled onto the wrong body.
Otherwise, a per-Domain default (DEFAULT_SITES, srb/core/ephemeris/env_support.py): equatorial lon-0 for Moon and Earth, Jezero crater (18.44°N, 77.45°E) for Mars.

env.ephemeris.site.* is deliberately not CLI-addressable. SiteSpec’s default is None, and the config-key extraction that backs both Hydra overrides and tab-completion (extract_defaults_from_class) only emits sub-keys for a field whose default is a populated nested config — a None default is a leaf with no children, the same reason BaseEnvCfg.ephemeris itself is declared as a non-None EphemerisCfg() default rather than None. There is therefore no env.ephemeris.site.lat_deg=… CLI override. To pin an explicit site, either point env.scenery at a real-terrain site that already carries the coordinates you want (precedence #2 above), or set it from Python:

from srb.core.ephemeris import EphemerisCfg, SiteSpec

cfg.ephemeris = EphemerisCfg(
    enabled=True,
    site=SiteSpec(body="moon", lat_deg=-89.66, lon_deg=0.0),
)

Radiometry. /World/sunlight’s intensity is not domain.light_intensity × state.irradiance_scale — state.irradiance_scale is an absolute, 1-AU-referenced (1 AU / r)², but Domain.light_intensity is already quoted at each body’s own mean distance (Mars = 590 W/m², a surface value at ≈1.524 AU), so multiplying the two double-counts the distance (Mars would land at ≈254 W/m², 43 % of the correct value). Instead:

intensity = domain.light_intensity
          × irradiance_multiplier(body, state.irradiance_scale)
          × sun_visibility_factor(elevation_deg, 0.5 * angular_diameter_deg)

irradiance_multiplier (srb/core/ephemeris/env_support.py) normalizes irradiance_scale by the body’s own mean-distance value, so the ephemeris contributes only the variation around Domain.light_intensity’s already-correct mean — ≈±3.4 % on Earth/Moon, ≈±19 % on Mars — not an absolute rescale. Cross-checked against a real kernel query: a Mars epoch near mean heliocentric distance yields ≈591 W/m² against Domain.MARS’s 590 (0.18 % off).

Offline fallback. Exactly like the terrain-lighting path above, the env path never queries or downloads when the pinned kernels are absent — it only ever probes for their presence on disk. on_missing="error" fails fast: the presence probe runs at config time regardless of whether the epoch is fixed or sampled, so a doomed run raises KernelsMissingError before a multi-minute Kit boot, not after (a sampled epoch also re-probes on every reset, in case kernels vanish between config time and a later reset). on_missing="fallback" (default) instead warns once — per domain for a fixed epoch, per site body for a sampled one — and leaves the sun exactly as it already is: the built-in static default for a fixed epoch, or whatever the previous reset left it at for a sampled one (no query, no write, and the epoch RNG is not even created/advanced on a fallback reset).

Limits.

Ephemeris on Domain.{MOON, MARS, EARTH} is the surface path described above. Domain.ORBIT is now also valid — see Environment Wiring (Phase 4). Domain.ASTEROID remains a config error (no ephemeris body exists for it).
Setting both env.sunlight_rpy and an enabled env.ephemeris is a config error — both would drive the sun’s orientation.
env.ephemeris.site.* is not CLI-addressable (above); the same is true of env.ephemeris.orbit.* (Phase 4).
env.ephemeris.drive_skydome is now honored — see Phase 4 below.
env.ephemeris.bodies, expose_observations, eclipse, epoch.time_scale, and epoch.update_interval_s are all consumed as of Phase 4 — see Environment Wiring (Phase 4).
A real terrain’s companion lighting now reaches a running environment — arbitrated against this ephemeris sun. As of terrain v2 Phase 4 (OI-1), BaseEnvCfg._add_scenery calls RealTerrain.attach_companions() automatically, and an enabled env.ephemeris.drive_sunlight wins: the companion lighting product is suppressed with a one-time warning naming both knobs. Without an ephemeris sun, an attaching companion lighting owns the scene sun (scene.sunlight = None, static sun randomizers skipped) — which is how a manifest lighting: "ephemeris" site finally takes effect in a running env. The full precedence table lives in Terrain Stacks → Companions and the ephemeris sun.
Orbit mode, a moving sun, eclipse, and observation terms are Phase 4 — see Environment Wiring (Phase 4) below. Per-env suns (as opposed to the single global /World/sunlight) remain out of scope.

Environment Wiring (Phase 4)

Phase 4 wires up the three EphemerisCfg knobs that Phase 3 accepted but did not consume — eclipse, expose_observations, drive_skydome — and extends env.ephemeris to Domain.ORBIT: a circular-orbit observer, analytic umbra/penumbra eclipse, a moving sun, phase-dependent earthshine, an opt-in observation vector, and deterministic skydome co-drive. All new numerics live in srb/core/ephemeris/orbit.py (pure numpy), with geometry additions in srb/core/ephemeris/geometry.py; the SpiceProvider backend itself is untouched.

Orbit mode (`Domain.ORBIT`)

EphemerisCfg.orbit: OrbitSpec | None (srb/core/ephemeris/orbit.py, default None) configures a circular Kepler orbit around the ephemeris site’s central body, with elements expressed in the central body’s J2000 axes (the scene world frame in orbit mode):

Field	Default	Meaning
`altitude_m`	required	Circular-orbit altitude above the body’s mean radius, meters. Must be `> 0`.
`inclination_deg`	`0.0`	Orbital inclination, degrees. Must be in `[0, 180]`.
`raan_deg`	`0.0`	Right ascension of the ascending node, degrees.
`arg_lat_deg`	`0.0`	Argument of latitude (angle from the ascending node to the observer) at the reference epoch — i.e. the orbit phase at `dt_s=0`.

Enabling ephemeris on Domain.ORBIT requires env.ephemeris.orbit to be set — EphemerisCfg(enabled=True) with orbit=None on an orbital domain is a config error (check_ephemeris_precedence), and conversely setting env.ephemeris.orbit on any non-Domain.ORBIT domain is also a config error (an OrbitSpec left over on a surface task would silently mean nothing, so it is rejected instead of ignored). The central body defaults to "earth" (resolve_orbit_site) and is overridable via an explicit env.ephemeris.site = SiteSpec(body=..., lat_deg=None, lon_deg=None) — an orbit-mode observer has no lat/lon; a surface SiteSpec (non-None lat_deg) is rejected the same way a stray body mismatch is on the surface path.

orbit_position_j2000(orbit, body, dt_s) propagates the observer’s position analytically (mean motion n = sqrt(GM / a³), a = body_radius + altitude_m, u = arg_lat_deg + n·dt_s, rotated into J2000 by R3(-raan) · R1(-inc)); the sun’s direction at that position is taken from the existing body-center SPICE query and applied directly (no offset for the observer’s LEO position) — the resulting sun-direction error is ~5×10⁻⁵ rad at LEO altitudes, negligible for lighting/eclipse cadence but not appropriate for navigation truth. GM_M3S2/BODY_RADIUS_M (orbit.py) cover earth/moon/mars only; other bodies raise.

Eclipse

EphemerisCfg.eclipse: bool (default False) multiplies the sun’s intensity by an analytic disk-overlap eclipse factor (eclipse_factor(sun, occluder) -> float, srb/core/ephemeris/geometry.py): 1.0 for no overlap, 0.0 for total occultation, 1 − (a_occ/a_sun)² for an annular (occluder’s disk fully inside the sun’s, smaller angular radius), and the circular-lens partial-overlap area formula otherwise. The occluder depends on domain:

Domain.ORBIT: the occluder is the orbit’s own central body, its state synthesized analytically from the propagated observer position (central_body_state) — no extra SPICE query.
Surface Domain.MOON: the occluder is "earth" (real solar eclipses as seen from the Moon) — "earth" is auto-added to the query bodies whenever eclipse=True on a moon surface site (query_bodies), even if not already listed in env.ephemeris.bodies.
Surface Domain.MARS/Domain.EARTH: no occluder is modeled; eclipse=True there is a no-op (factor stays 1.0) with a one-time warning per site body — Mars’ moons and Earth’s own lunar/solar eclipse geometry from an Earth surface site are both out of scope.

env.ephemeris.bodies must always include "sun" (EphemerisCfg.validate enforces this — sun_light_params/orbit_sun_light_params always look it up); listing other bodies (e.g. "earth") explicitly is only needed when you want their geometry beyond what eclipse auto-adds.

Moving sun

EpochSpec.time_scale (default 0.0, must be >= 0.0) turns on a moving sun: whenever time_scale > 0, an interval-mode event term progress_sun_ephemeris (srb/core/mdp/events.py, registered by BaseEventCfg._update_sunlight) fires every epoch.update_interval_s seconds (default 600.0) of simulation time and re-applies the sun at epoch0 + time_scale * elapsed_sim_time, using the exact same sun_light_params/orbit_sun_light_params math as the fixed/sampled paths. epoch0 is the fixed utc or, under a sampled utc_range, the most-recently-reset-sampled epoch — a sampled epoch and a moving sun compose: each reset draws a new epoch0, and the interval term advances from there until the next reset. A shift that walks the epoch outside the pinned kernel bundle’s de440s.bsp coverage window (1850-01-01 to 2149-12-31) raises the same ValueError _parse_epoch always raises for an out-of-range epoch — a very large time_scale over a long episode can hit this.

Orbit + eclipse with time_scale=0 freezes the eclipse factor. Reset appliers use dt_s=0.0 for the observer’s orbital position, so without a moving sun (time_scale > 0) the spacecraft never advances along its orbit within an episode: the eclipse factor computed at reset is constant — pinned at whatever arg_lat_deg says the orbit phase is at that reset — for the entire run, rather than beating in and out of shadow as a real orbit would. Enable epoch.time_scale > 0 if an eclipse beat during the episode is wanted.

Phase-dependent earthshine

The terrain-lighting earthshine term (Moon sites; see Terrain lighting above) now scales its 200 lux base by the Earth-as-seen-from-the-Moon illuminated fraction f = (1 + cos φ) / 2, where φ is the phase angle at Earth between the Earth→site and Earth→sun directions (earthshine_fraction(sun, earth), srb/core/ephemeris/geometry.py) — full-Earth (“new Moon” from the lunar surface) yields f ≈ 1, new-Earth (“full Moon”) yields f ≈ 0, rather than the previous constant-200-lux approximation. The negative-sun-elevation clamp (max(elevation_deg, 0.0)) is also removed from the same lighting path: sun_visibility_factor already zeroes intensity below the horizon, so downstream consumers now see the true (possibly negative) elevation instead of a clamped 0.0. This earthshine term lives only in the terrain-lighting bake path (srb/core/ephemeris/lighting.py) — the env.ephemeris env path has no earthshine light prim; only /World/sunlight is driven there.

Observation vector

env.ephemeris.expose_observations: bool (default False) adds an obs["ephemeris"] key of shape (num_envs, 7) to the Direct observation dict (srb/core/env/common/base/direct/impl.py, wired centrally — not per-task). Slot order:

Slots	Content
`0:3`	Sun unit direction, rotated from world into the robot’s base frame (xyzw quaternion inverse rotate).
`3:6`	Occluder unit direction, base frame; all-zero when there is no active occluder (eclipse disabled, or a surface domain/site with none modeled).
`6:7`	Eclipse factor, broadcast scalar; `1.0` whenever eclipse is disabled.

The term (ephemeris_observation, srb/core/ephemeris/observations.py) is pure tensor math over the same _ephemeris_state/_ephemeris_occluder/ _ephemeris_eclipse_factor stash every ephemeris application site (config- time fixed-epoch, per-reset, and per-interval) writes onto the unwrapped env — it does not call SPICE and does not run any USD/Isaac code, so it is cheap per-step. Note it bypasses the observation-delay and non-finite-scrub wrappers that other observation terms go through (see the Task-7 review note in the Phase 4 plan) — obs["ephemeris"] is written directly into the assembled observation dict after those wrappers have already run over the rest of the terms (merged post-assembly), so an observation_delay config or the non-finite scrub applied to other terms never touches it: it is always the current-step value, and a NaN there is never scrubbed to 0.0 the way a blown-up proprio term would be.

Skydome co-drive

env.ephemeris.drive_skydome: bool (default False) is now honored on surface domains (it remains inert on Domain.ORBIT, which has no skydome). When both drive_sunlight and drive_skydome are True, the skydome’s world yaw is set deterministically to -azimuth_sun (the sun’s reported azimuth, in the same grid-corrected frame north_yaw_deg already folds in) every time the sun itself is applied — config time, per-reset, and per- interval alike — and randomize_skydome_orientation is nulled for the same reason the sun randomizers are nulled in Phase 3 (leaving it enabled would clobber the deterministic yaw on its next interval tick). Setting drive_skydome=True without drive_sunlight=True is now a config error (check_ephemeris_precedence) rather than a silently inert combination — the co-drive logic lives inside the shared sun-apply path, so it is unreachable without drive_sunlight.

Absolute texture alignment is not, and cannot be, guaranteed. There is no per-texture metadata recording which real-world cardinal direction a skydome texture’s features point at, so “the skydome’s bright region lines up with the real sun” is not a claim this feature makes. What Phase 4 delivers is determinism and co-motion: the same epoch always yields the same dome yaw, and the dome visibly turns in lockstep with the sun (moving-sun mode, per-reset resampling) rather than staying frozen at its domain-default orientation while the sun moves independently.

CLI-addressability caveat

Exactly like env.ephemeris.site.* in Phase 3, env.ephemeris.orbit.* is not CLI-addressable. OrbitSpec’s field default on EphemerisCfg is None, and the config-key extraction backing both Hydra overrides and tab-completion only emits sub-keys for a field whose default is a populated nested config — a None default is a leaf with no children. There is therefore no env.ephemeris.orbit.altitude_m=… CLI override; set it from Python:

from srb.core.ephemeris import EphemerisCfg
from srb.core.ephemeris.orbit import OrbitSpec

cfg.ephemeris = EphemerisCfg(
    enabled=True,
    orbit=OrbitSpec(altitude_m=400_000.0, inclination_deg=51.6),
    eclipse=True,
)

Limits

Circular-orbit approximation. No J2 (oblateness) perturbation, no drag, no eccentricity — a real LEO orbit’s node/argument-of-latitude drift and altitude decay are not modeled. This is a lighting/eclipse-cadence model, not a navigation-truth one.
Sun direction taken from the body center, not offset for the observer’s LEO position — ~5×10⁻⁵ rad error at LEO altitudes (negligible for lighting, stated explicitly rather than silently absorbed).
Disk-overlap eclipse only — no atmosphere, no limb darkening, no penumbra gradient within the partial band beyond the geometric lens-area fraction.
Skydome co-drive is deterministic, not absolutely aligned (above).
Earthshine phase-scaling is terrain-lighting-only — the env path drives no earthshine light prim (above).
env.ephemeris.orbit.* is not CLI-addressable (above).
The observation term bypasses the observation-delay and non-finite-scrub wrappers (above).
An optional visual Earth/Moon prim in orbit mode (mentioned as a stretch goal in the Phase 4 outline) was deliberately dropped from this phase.

Illumination Products (Phase 5)

Phase 5 adds a kernel-free "table" EphemerisProvider (srb/core/ephemeris/table_provider.py, registry key "table", reserved since Phase 1) that synthesizes EphemerisState from a precomputed illumination product — sun (and, optionally, Earth) geometry rasterized onto a terrain-patch grid ahead of time by an external tool (lunarlab’s spice_maps, or PGDA/USGS PSR products) — instead of a live SPICE query. Everything downstream of query() (the env sun/skydome drive, radiometry, eclipse plumbing) is unchanged: provider="table" is just a different EphemerisState source, so the whole Phase 3/4 env path runs with zero NAIF kernels.

Schema `srb_illum/2`

srb/core/ephemeris/products.py defines the on-disk product format: a single np.savez_compressed archive (SCHEMA_ID = "srb_illum/2", product_version = 2) holding a meta JSON string plus rasters on a north-up, axis-aligned, pixel-center- sampled grid (ProductGrid) centered on a terrain patch — the same grid convention Real-Terrain-v2 bakes use. Two tiers, at least one required:

Tier	Key	Dtype/shape	Notes
epoch (all-or-none)	`sunlit`	`bool (T,H,W)`	Per-epoch boolean line-of-sight mask.
epoch	`sun_fraction`	`float32 (T,H,W)`, NaN holes	Continuous illuminated-disk fraction.
epoch	`times_utc`	`(T,)` ISO-8601 strings	Epoch axis.
epoch	`sun_azel`	`(T,2) float64`	`[true-north compass azimuth deg, patch-center elevation deg]`.
epoch	`sun_angular_diameter_deg`	`(T,) float64`	Per-epoch sun angular diameter.
epoch, optional	`earth_los`	`bool (T,H,W)`	Present only when the source carries Earth visibility.
epoch, optional	`earth_azel`	`(T,2) float64`	Present only when the source carries Earth geometry.
static, per-key optional	`psr`	`uint8 (H,W)`, `{0, 1, 255=nodata}`	Permanently-shadowed-region mask.
static, per-key optional	`lit_fraction`	`float32 (H,W)`, NaN holes	Time-averaged illuminated fraction.
static, per-key optional	`max_sun_el`	`float32 (H,W)`, NaN holes	Maximum sun elevation ever reached at that cell.

Legacy srb_illum/1 archives are not readable: load_product rejects them with legacy schema 'srb_illum/1' requires explicit migration, so re-import the source product instead of loading an old sidecar.

IlluminationProduct.validate() (called by both save_product/load_product) enforces the all-or-none rule on the epoch group and rejects a product with neither tier present. The epoch tier’s sunlit/sun_fraction (and, on the grid mode below, earth_los) share one _sample() core: bilinear for float rasters (with degenerate-axis fallback for a 1×N/N×1 grid), nearest for bool/uint8 masks, PSR_NODATA (255) returned out-of-bounds for psr.

ProductGrid.patch_center_xy_m anchors patch-local (x, y) meters (x=grid- east, y=grid-north, origin at the patch center) to the grid’s absolute pixel transform; local_xy_to_rowcol_f/rowcol_to_local_xy are exact inverses of each other, which is what makes illumination_fraction_at’s round trip through env.scene.env_origins correct (below).

Azimuth/elevation conventions (P5-D1/D2/D6). sun_azel/earth_azel are stored already corrected to true-north compass azimuth and patch-center elevation — i.e. in the exact convention query()’s BodyState.azimuth_deg/ elevation_deg use, so TableProvider.query() can hand them straight to geometry.direction_world() with no further transform (only site.north_yaw_deg, the same grid-north correction the SPICE path applies, is folded in at query time). The correction from the source’s map-grid azimuth and off-center elevation to this convention happens once, at import time (below), not at query time.

Importers

srb/core/ephemeris/importers.py converts three external formats into srb_illum/2, always reprojecting onto a north-up target ProductGrid and applying the map-grid → true-north/site azimuth-elevation correction lunarlab’s center_azel convention requires: az_true = az_grid − γ (γ = grid convergence at the site, from srb.terrain.ingest.planetary_crs.north_convergence_deg) plus a curvature- ramp elevation correction for the parallax between the source grid’s own center and the target patch center (el_site = el + degrees(s_par / body_radius_m), s_par the along-azimuth component of the center-to-center offset — a cell displaced toward the sun azimuth sees a higher sun, matching lunarlab’s own masks.ramp_elevation). All heavy dependencies (zarr, rasterio, pyproj) import lazily inside functions, so importing importers.py itself stays Isaac-free.

Source	Function	Notes
lunarlab `spice_maps` `epochs.zarr`	`import_lunarlab_zarr`	Requires `zarr`/`numcodecs`.
lunarlab legacy `save_masks` `.npz`	`import_lunarlab_npz`	No extra dependency beyond numpy.
PGDA product-69 LPSR GeoTIFF	`import_pgda_lpsr`	Requires `rasterio` (a base SRB dependency, not extra). Static-only (no epoch tier).

Every importer takes exactly one of two target-resolution modes (_check_target_xor_baked_dir), plus a --body {moon,mars,earth} flag (default moon, threaded straight into the importer’s body= kwarg) naming the body the target terrain patch sits on:

--baked-dir mode — target grid, CRS, patch center, and site lat/lon are all read from a baked Real-Terrain-v2 patch’s meta.json (target_grid_from_baked_dir); the output defaults to <baked-dir>/illumination.npz and a illumination block (file name, source label, epoch count/step, static keys present, import timestamp) is recorded into the same patch’s cache meta.json under a cache_lock. --lat/--lon are rejected in this mode (SystemExit(2)) since the bake’s meta.json is the sole georef authority — they cannot be overridden alongside --baked-dir.
Explicit-target mode (--crs/--center-xy/--size-m/--gsd-m/ --lat/--lon/--out) — no baked patch required; useful for importing a product ahead of a bake, or onto a grid that has nothing to do with a terrain cache. --size-m must be an integer multiple of --gsd-m (SystemExit(2) otherwise, naming both values and the remainder) — a non-integer ratio would skew the patch center by a sub-pixel offset.

--epoch-range/--epoch-stride (default stride 1) are rejected (SystemExit(2)) together with --source pgda-lpsr: that product is static-only and carries no epoch axis, so the flags have nothing to apply to.

CLI examples (full reference: srb ephemeris):

# --baked-dir mode: grid/CRS/site inferred from an already-baked patch
srb ephemeris import --source lunarlab-zarr \
  --input /data/lunarlab/shackleton_240m/epochs.zarr \
  --baked-dir .cache/terrain/moon/shackleton_rim/<patch-key>

# Explicit-target mode: no baked patch, target grid given directly
srb ephemeris import --source pgda-lpsr \
  --input /data/pgda/lpsr_shackleton.tif \
  --crs "IAU_2015:30135" --center-xy 0.0,10309.969398639505 \
  --size-m 4096 --gsd-m 5.0 --lat -89.66 --lon 0.0 \
  --out /tmp/shackleton_illumination.npz

The zarr dependency. srb[spice] now pulls in zarr>=2.16,<4 and numcodecs (needed only by import_lunarlab_zarr); the pyproject.toml extra and uv.lock were updated together. The dev container’s image-baked venv predates this change and does not have zarr installed today — the Dockerfile runs uv sync --frozen --extra all (Dockerfile:466), so the next image rebuild picks it up; until then, import_lunarlab_zarr raises a clear ImportError naming the extra rather than an opaque ModuleNotFoundError (import_lunarlab_npz/import_pgda_lpsr need no such extra — rasterio is already a base dependency).

Decode gotchas, all copied honestly from the source formats rather than silently reinterpreted:

Gotcha	Detail
Bit-packed masks	lunarlab zarr’s `sunlit`/`earth_los` are packed along the last axis; decoded via `np.unpackbits(arr, axis=-1)[..., :W]`.
254-scaled fraction	lunarlab zarr’s `sun_fraction` is `uint8`; decoded as `q / 254.0` with sentinel `q == 255 → NaN`.
Map-grid azimuth	Source az/el are in the source grid’s own map convention, not true-north/patch-center; corrected at import time (above), not left for the caller to reinterpret.
Curvature-ramp elevation	The elevation correction for the source-grid-center → target-patch-center offset (above) is a first-order parallax approximation, not a full spherical recomputation.
`earth_los` semantics	Lunarlab’s `earth_los` is a DSN-union visibility mask (true for the union of Deep Space Network station view cones), not a strict single-point Earth-center line-of-sight — SRB imports this field verbatim and does not reinterpret or rename it; treat it as “Earth-observable by some real antenna network,” not “Earth geometrically above the local horizon.”
PSR DN rule	PGDA LPSR `int16` rasters: PSR iff `DN == 20000` exactly, nodata iff `DN == -32768`; a `uint8` canonical input `{0, 1, 255}` is passed through unchanged; any other dtype raises `ValueError`.
No fractional PSR decode	Neither lunarlab nor PGDA’s `psr`/LPSR sources carry a continuous shadow-fraction value — `psr` is always a hard `{0, 1, 255}` mask, never interpolated to a probability.

Table provider

TableProvider.query() (srb/core/ephemeris/table_provider.py) resolves a product file via set_table_product(path) (wins) or the SRB_EPHEMERIS_TABLE_PRODUCT environment variable (fallback), caching at most one loaded IlluminationProduct resident at a time (mtime-keyed, so a re-imported file on disk is picked up without a process restart). It:

Requires the epoch tier. _load() raises ValueError for a static- only product (“the table provider needs per-epoch sun az/el”) — a PSR-only PGDA import can drive masked spawn/reward via illumination.py (below) but cannot itself back the "table" provider’s query().
Resolves the nearest epoch via IlluminationProduct.epoch_index() with a tolerance of half the product’s own epoch step (or 3600 s if the product has fewer than two epochs); a query epoch further than that raises ValueError rather than silently interpolating.
Surface sites only. if not site.is_surface: raise ValueError(...) — the table provider has no orbit-mode support (no central-body-relative geometry in the schema). As of this task, EphemerisCfg.validate() rejects provider="table" combined with env.ephemeris.orbit set at config time (srb/core/ephemeris/config.py), rather than leaving the identical rejection to the first query/reset after a Kit boot.
epoch_et is approximate. EphemerisState.epoch_et is filled via approx_et_seconds() (P5-D4): a fixed TT − UTC = 69.184 s offset (32.184 s + the 37 leap seconds in effect since the last leap second, 2016-12-31), valid to <2 ms error for any epoch from 2017 onward — but wrong before 2017 and requiring an update if a future leap second is ever inserted. Nothing downstream numerically consumes epoch_et today (recorded, not load-bearing).
Bypasses the kernel-presence gate. The shared reset-event prologue (_resolve_sun_event_context, srb/core/mdp/events.py) explicitly skips kernels_present() when ephemeris_cfg.provider == "table" — the whole point of this provider is running with zero kernels on disk, so gating it on kernel presence would defeat itself.
Terrain-lighting bake path stays SPICE-only. The Phase 2 terrain- lighting bridge (srb/terrain/lights/ephemeris_bridge.py) always calls query() with no provider= argument, which defaults to "spice" (srb/core/ephemeris/provider.py) — a manifest site’s lighting: "ephemeris" bake is not (yet) table-provider-addressable; only the env.ephemeris env path can use "table".
eclipse=True + provider="table" is data-dependent, not rejected. query_bodies() still adds "earth" to the query bodies whenever eclipse=True on a Moon surface site, exactly as with "spice". Whether this then does anything depends on the loaded product: when it carries earth_azel (as lunarlab products do), TableProvider.query() builds a real per-epoch Earth BodyState (direction from the corrected az/el, angular diameter from a fixed Earth mean distance) and the eclipse factor computes normally; when it does not (e.g. a PGDA-only or static-only product — although a static-only product cannot back "table" at all, above), the Moon-site occluder lookup silently returns None and the eclipse factor stays pinned at 1.0 for the whole run. A warning is emitted once per (body, provider) the first time this happens (srb.core.ephemeris.env_support.warn_eclipse_occluder_missing_once, called from _apply_sun_state in srb/core/mdp/events.py) — the analogous Mars/Earth “no occluder modeled” case (Phase 4) already warned once via its own occluder_body is None guard, which never fires here because surface_occluder_body("moon") is "earth" (not None); this is a second, independent warn-once path for exactly that gap. The factor still pins at 1.0 regardless — the warning is diagnostic only. This was investigated for this task and left as a documented limit rather than a blanket EphemerisCfg.validate() rejection: rejecting eclipse=True outright for every provider="table" config would also block the working, intended case (a lunarlab product that does carry Earth geometry) — the config layer has no way to inspect the product file’s contents (it is resolved lazily, independently of EphemerisCfg construction), so there is no config-time signal to gate on. Import a product with Earth geometry if a working table-provider eclipse is wanted; otherwise expect a once-warned no-op.

Consumers

srb/core/mdp/illumination.py bridges the schema into two MDP-facing surfaces, both operating on the static tier only (the epoch/time-cube tier is not wired to any reward/observation consumer as of this phase — see Limits below):

Term/helper	Kind	Params (non-exhaustive)	Behavior
`reset_root_state_uniform_illumination_masked`	reset-mode event term	`pose_range`, `velocity_range`, `asset_cfg`; `mask` is `"psr"` (default) or `"lit_fraction"`; `threshold` defaults to `0.5`; `invert` defaults to `False`; optional `product_path` override	Spawns `asset_cfg` at an `(x, y)` uniformly drawn from candidate grid cells passing the mask/threshold test (jittered `±gsd/2`), with `z`/`roll`/`pitch`/`yaw`/velocities sampled exactly like `reset_root_state_uniform_poisson_disk_2d`. `mask="psr"` selects cells with `psr == 1` (nodata `255` always excluded); `mask="lit_fraction"` selects cells with `lit_fraction >= threshold` (NaN excluded); `invert=True` complements the selection within valid cells. Raises `ValueError` if the candidate set is empty.
`illumination_fraction_at`	torch helper	`positions_w`; `key` is `"lit_fraction"` (default) or `"max_sun_el"`; optional `product_path` override	Samples a float static raster at world positions (converted to patch-local via `env.scene.env_origins`), NaN → `0.0`, safe to plug directly into a reward term. Restricted to the two float rasters — `psr`’s out-of-bounds sentinel is `255`, not NaN, so it is deliberately excluded here; sample `psr` directly via `IlluminationProduct.sample_static` for that case.

Both resolve the product via resolve_illumination_product(): an explicit product_path argument wins, else env.unwrapped.cfg._scenery.illumination_product_path (a RealTerrain property, below) — raising ValueError naming srb ephemeris import as the remediation when neither resolves.

utc_windows + sunlit_epoch_windows end-to-end (product → windows → cfg). EpochSpec.utc_windows: tuple[tuple[str, str], ...] | None (Phase 5 Task 6) is a third, mutually-exclusive alternative to utc/utc_range: a set of duration-weighted epoch windows sampled per reset by sample_epoch_windows() (a window is picked with probability proportional to max(duration_s, 1.0), then an epoch drawn uniformly inside it — a single-epoch window, start == end, gets weight 1.0 rather than 0 so it stays sampleable). srb.core.ephemeris.products.sunlit_epoch_windows() derives such a tuple directly from a loaded product — either grid mode (fraction of valid, non-NaN cells with sunlit set across the whole patch) or point mode (xy_local given — the continuous sun_fraction value at that point) — merging consecutive passing epochs into (start_utc, end_utc) windows:

from srb.core.ephemeris import EpochSpec
from srb.core.ephemeris.products import load_product, sunlit_epoch_windows

product = load_product("illumination.npz")
windows = sunlit_epoch_windows(product, min_lit_fraction=0.6)  # grid mode

cfg.ephemeris.epoch = EpochSpec(utc=None, utc_windows=windows)

The tuple’s shape ((start, end) ISO-string pairs) feeds EpochSpec.utc_windows with no conversion — this is pinned by a dedicated shape- compat unit test (test_epoch_windows_shape_compat_with_sunlit_epoch_windows, tests/unit/test_ephemeris_config.py). Note (Task 6 reviewer finding): EpochSpec.time_scale > 0 (the Phase 4 moving-sun knob) composes with utc_windows the same way it composes with utc_range — each reset still draws a fresh epoch0 from the window set, but the moving-sun interval term then walks the epoch forward from there with no further reference to the window boundaries, so a large time_scale over a long episode can walk the applied epoch well outside the curriculum window that was actually sampled.

Limits

The precomputed sidecar is not part of CacheKey (srb/terrain/cache.py). Re-importing a product over an existing illumination.npz at a fixed cache key is invisible to the terrain bake cache — this is by design (the sidecar is a separate out-of-band artifact, not a bake input), but it means a stale sidecar is not detected or invalidated automatically the way a changed DEM/colour stack would be.
Per-env location variety (K>1) uses the primary location’s sidecar only. RealTerrain.illumination_product_path (srb/assets/scenery/real_terrain.py) always derives from primary_baked_usd_path — the index-0 / requested-center child of a multi-location variant set (Terrain v2 Phase 5a) — never from any of the other K−1 per-env locations. A masked-spawn/reward term therefore reads the same illumination raster regardless of which of the K locations a given env actually landed on.
Leap-second caveat for imported ladders. approx_et_seconds’s fixed 69.184 s TT−UTC offset (above) is only valid for epochs from 2017-01-01 onward; an imported product with pre-2017 epochs, or any future epoch after a new leap second is inserted (none since 2016-12-31, per naif0012.tls’s own “current through the last leap second” caveat in the kernel bundle table), needs this constant revisited.
earth_los semantics are inherited, not redefined. As noted in the decode-gotchas table above, lunarlab’s earth_los is a DSN-union visibility mask, not strict geometric line-of-sight from the site point; SRB imports and samples it as-is.
eclipse=True + provider="table" is data-dependent (above) — works when the product carries Earth geometry, otherwise no-ops (factor pinned at 1.0) with a warning emitted once per (body, provider) at first application. provider="table" + env.ephemeris.orbit set is rejected at config time (above); the table provider has no orbit-mode support.
The terrain-lighting bake path (lighting: "ephemeris" in the terrain manifest) remains SPICE-only — it always queries with the default provider="spice"; Phase 5’s "table" provider is env-path-only.
The epoch/time-cube tier has no reward/observation consumer yet. illumination_fraction_at/the masked-spawn term both read only the static tier (psr/lit_fraction/max_sun_el); nothing in srb/core/mdp/illumination.py (or elsewhere) samples the per-epoch sunlit/sun_fraction cube for a reward/observation term — that epoch-indexed data is consumed only by the "table" provider’s own query() (to synthesize sun geometry) and by sunlit_epoch_windows (to derive utc_windows), both at the config/curriculum level, not per-step. A future per-step “instantaneous predicted illumination at time t” reward is deliberately deferred.

Memory and Threading Contract

spiceypy is imported in exactly one place: spice_provider.py. No other module in SRB — core, CLI, tests outside the ephemeris suite, or otherwise — should import spiceypy directly. This keeps srb --help, tab-completion, and parser construction Isaac-free and import-light, and keeps the optional dependency isolated to a single, easily-mocked seam.
SPICE’s C kernel pool is global, mutable, and not thread-safe across independent furnish/query cycles. SpiceProvider serializes access with a module-level lock and furnishes the kernel set at most once per process (_FURNISHED guard) — repeated query() calls reuse the already-loaded pool rather than re-furnishing.
Forked worker processes (e.g. pytest-xdist, multiprocessing data loaders) each inherit — or, after an actual fork(), share low-level SPICE state in ways that are not safe to rely on. Call srb.core.ephemeris.spice_provider._reset_for_tests() (clears SRB’s _FURNISHED flag so the next query re-furnishes) and/or spiceypy.kclear() (clears the underlying CSPICE kernel pool itself) at the start of any new worker/process that will use the ephemeris subsystem, and in test fixtures that need a clean furnish state between cases.

Environment Configuration — Terrain Stacks

Real-terrain v2 composes one simulation-ready terrain from multiple real data products — a coarse gap-free DEM base, finer partial-coverage DEM insets, sunlit and permanently-shadowed-region (PSR) colour mosaics, masks, and quality layers — instead of the one-DEM/one-ortho pairing of the v1 manifest sites. This page is the operator-facing guide to that stack model: the per-body source catalog, arbitrary lat/lon region requests (RealTerrain(body, lat_deg, lon_deg, size_m) and srb asset download terrain --lat --lon --size-m), the deshade knob, the consent gate, and how a real terrain’s companion assets (sky / lighting / rocks) are arbitrated against the ephemeris sun when both want to own the scene.

For the fusion engine itself (blending, provenance, budgets) see Real Terrain Assets; for authoring catalog layers and v2 manifest sites see Adding Terrain Sites.

Read this first — the shipped catalogs now carry real, curated data, but coverage is still narrow. The v2 request path is implemented and now proven end to end against real network products, not just a synthetic fixture: srb/terrain/catalog/moon.yaml carries 19 real, sha256-pinned layers (2 LOLA polar DEM bases, 4 NAC DTMs, 1 Haworth SfS DEM, 4 SfS A3CLR DEM/ortho, 1 ShadowCam DTM + 1 cmosaic, 1 NAC ROI, 1 WAC_EMP colour mosaic, 1 LOLA LDAM 1064 nm colour map, 1 PGDA LPSR mask, 2 NAC_POLE_SOUTH_CM reference tiles); mars.yaml carries 2 (a HiRISE Jezero DTM + ortho); earth.yaml carries 2 (one Copernicus GLO-30 polar tile, one 3DEP 1 m non-polar tile). Note that access: cog_remote remains a catalog compatibility label. Production no longer gives its URL directly to GDAL: the v2 loader downloads the complete source through SRB’s redirect-validating, size-bounded provider, verifies its pinned SHA-256, then asks rasterio to read a local window. Consequently every contributing production layer records bytes_verified: true. This closes an allowlisted-origin redirect escape at the cost of first-bake download size and latency. Four real region bakes — Moon polar, Mars Jezero, Earth polar, Earth non-polar — complete end to end with area_fraction=1.0 on every DEM base layer. That claim is scoped to DEM bases deliberately: the per-product mask DN decoding shipped later on this branch moved the Moon-polar colour base off 0.0, but not to 1.0 either, so no single figure covers both kinds of base. See Catalog status per body below and docs/research/real_terrain_v2/ for the full §6.11 validation study run against this data. Coverage is still narrow, and a request outside every curated footprint still resolves to an empty plan: in an environment that degrades to the procedural terrain with a one-time warning naming the reason; on the CLI it is a clear error (exit 1). LDAM is now curated as imbrium_ldam_50s_1000m: its detached PDS3 .LBL/.IMG pair was fetched over HTTPS, independently hash/size verified, footprint/radiometry reviewed, and used in two deterministic real 8 km colour bakes. WAC_EMP 643 nm remains priority 3 and therefore precedes priority-4 LDAM in their overlap; LDAM is 1064 nm active-laser albedo, so a spectral seam can remain and no gain is invented. The contribution bake used an isolated LDEM+LDAM catalog because no shipped Moon DEM covers the 50S–60S annulus where LDAM would otherwise be the sole colour source. See the catalog header and durable P2-05 evidence. The NAC_POLE_SOUTH_CM_065/CM_135 mosaics were blocked on the same list for their per-tile size until 2026-08-02, when the one band-1 tile covering Connecting Ridge was downloaded and pinned per epoch (lroc_nac_pole_south_cm065_p892s2250_1m and its cm135 sibling, 8,276,814,528 B each, allow_large_download: true plus reference_only: true, so they are never resolved as bake input at all — only by explicit layer_id); the remaining tiles stay unpinned, an operator decision per site. (A third family, SfS A3CLR, was blocked for the same allowlist reason until 2026-08-02, when the allowlist was deliberately widened and its Connecting Ridge / Haworth DEM + orthomosaic layers were pinned for real from the Zenodo archive.) SRB_TERRAIN_CATALOG_DIR still lets you swap in a hand-built or fixture catalog for testing.

The catalog model

A per-body source catalog (srb/terrain/catalog/<body>.yaml, loaded by srb.terrain.catalog.loader) is the reviewed, pinned record of every real data product v2 may touch: URL(s), sha256, footprint, native GSD, license, priority_class, and the curated half of the large-download consent gate. Given a region, the resolver (srb.terrain.catalog.resolve.resolve_stacks) intersects the catalog’s footprints with the requested patch and emits a CompositionPlan — a prioritized DEM stack plus sunlit-colour, PSR-colour, mask, and quality stacks — which the fusion engine bakes into one terrain. Preview a plan without moving a byte:

srb asset stack --body moon --lat -89.46 --lon 137.3 --size 2000
srb asset stack --body moon --lat -89.46 --lon 137.3 --size 2000 --json

srb asset stack also prints whether deshading would apply (deshade: on (request) / off) — note the printed status reflects only the --deshade flag; a manifest site’s own deshade: true can still force it on for a site bake (see Deshade below).

There are two ways to consume the catalog:

A declared-stack manifest site — a srb/terrain/manifest.yaml entry (manifest version: 2) that names which catalog layers to fuse (dem_stack/color_stack). Bakes through the same RealTerrain(body=..., site=...) / srb asset download terrain <site> calls a v1 site uses.
An arbitrary region request — no manifest entry at all; the resolver finds coverage by footprint intersection. The rest of this page is mostly about this form.

Catalog status per body

Body	Layers	Curated families
moon	19	`pgda_ldem_83s_10mpp` / `pgda_ldem_80s_20mpp` (LOLA polar DEM bases); `lroc_nac_dtm_shackrdge02` / `nobile03` / `malapert02` / `malapert03` (NAC DTMs); `usgs_astro_haworth_sfs_dem_1m` (Haworth SfS DEM); `zenodo_sfs_a3clr06_connridge_dem_5m` / `zenodo_sfs_a3clr06_connridge_omos_5m` / `zenodo_sfs_a3clr02_haworth_dem_5m` / `zenodo_sfs_a3clr02_haworth_omos_5m` (SfS A3CLR DEM + orthomosaic, Connecting Ridge and Haworth); `lroc_shadowcam_dtm_faustini_6m` + `lroc_shadowcam_cmosaic_faustini01_20m` (ShadowCam DTM + cmosaic); `lroc_nac_roi_haworth_lo1_20m` (NAC ROI); `lroc_wac_emp_643nm_south_100m` (WAC_EMP 643 nm colour); `imbrium_ldam_50s_1000m` (LOLA LDAM 1064 nm normal albedo, nominal 1000 m at the pole, `cap:50S`); `pgda_lpsr_85s_60m` (PGDA LPSR mask); `lroc_nac_pole_south_cm065_p892s2250_1m` + `lroc_nac_pole_south_cm135_p892s2250_1m` (NAC_POLE_SOUTH_CM band-1 tile `P892S2250`, one per subsolar-longitude epoch — `reference_only: true`, so resolvable only by explicit `layer_id`, never as bake input)
mars	2	`usgs_astro_hirise_jezero_dtm_1m` + `usgs_astro_hirise_jezero_ortho_1m` (HiRISE Jezero DTM + ortho)
earth	2	`copernicus_glo30_dryvalleys_s78e161` (Copernicus GLO-30 polar); `usgs_3dep_nm_southeast_x45y364_1m` (3DEP 1 m non-polar)

This is a curated subset, not exhaustive global coverage — a request whose footprint isn’t covered by any of the layers above still falls back to the v1 single-source path (or, for a region request, an empty plan). The moon starter-set spec targeted ~15 layers. LDAM is now pinned as imbrium_ldam_50s_1000m: a detached PDS3 .LBL/.IMG bundle from the HTTPS-only imbrium provider, with independent file hashes and sizes. Its cap:50S footprint is the inscribed extent, and its nominal 1000 m/pixel resolution is true at the pole. WAC_EMP’s priority 3 keeps its 643 nm product ahead of priority-4 LDAM where both cover; consumers must tolerate the possible 643 nm/1064 nm spectral seam. NAC_POLE_SOUTH_CM_065/CM_135 left that list on 2026-08-02: the one band-1 tile covering Connecting Ridge (P892S2250) is now pinned per epoch as a reference_only: true comparison reference; the remaining tiles stay unpinned, an operator decision per site — the 2026-08-02 download re-measured the host at 8.2–8.4 MB/s (~17 min for the 8.28 GB band-1 tile), not the ~1.3 MB/s the original curation saw, so per-tile size (8.3–31.2 GB) rather than throughput is what makes each one a decision. See Adding Terrain Sites → Adding a v2 source layer for the authoring workflow to curate more layers.

Providers

Every catalog layer’s provider is a closed, exact-hostname allowlist — srb/terrain/sources/*.py. Phase 5b adds three Mars/Earth providers on the same public-bucket pattern as the existing ones, and imbrium (2026-08-02) is listed alongside them:

Provider	Body	Hosts	Auth
`uahirise`	mars	`hirise.lpl.arizona.edu`, `www.uahirise.org`, `uahirise.org`	none
`copernicus_s3`	earth	`copernicus-dem-30m.s3.amazonaws.com`, `copernicus-dem-90m.s3.amazonaws.com`	none
`usgs_3dep`	earth	`prd-tnm.s3.amazonaws.com`	none
`imbrium`	moon	`imbrium.mit.edu`	none

A fourth, imbrium (imbrium.mit.edu, the LOLA team’s MIT archive, moon, no auth), was registered on 2026-08-02. It is the only provider that accepts https only: the archive is linked from PGDA over plain http://, but the host was probed and serves valid TLS, so the plaintext scheme is refused rather than transcribed — and since the fetch path re-checks _accepts_url on every redirect hop, that also turns an https → http downgrade redirect into a hard error. The curated LDAM layer uses this provider (see the LDAM note above).

All three Phase 5b providers are plain HTTPS GETs against public, unauthenticated hosts/S3 buckets — there is no credential plumbing anywhere in the fetch path (no API keys, no signed URLs, no AWS SigV4). This is a real limitation, not just an unimplemented convenience: a product that requires authenticated access (a private bucket, an API-key-gated endpoint) cannot be curated into the catalog today regardless of license, until credentialed access is added. Existing Mars USGS products (CTX, HRSC-MOLA) need no new provider — the existing usgs_astro hosts already cover them.

Region requests

from srb.assets.scenery import RealTerrain

terrain = RealTerrain(body="moon", lat_deg=-89.66, lon_deg=0.0, size_m=2000.0)

lat_deg / lon_deg / size_m are all-three-or-none — setting any one without the others is a validation error naming the missing field(s) — and mutually exclusive with both site= and dem_path= (and with the patch-override fields patch_size_m/patch_center_xy_m: a region’s own size_m and resolved center define the bake footprint). lod maps through; size_m may be at most 8000 m (MAX_REGION_SIZE_M — tile larger areas as multiple requests).

Because these are plain numeric fields (not a nested config), the intent is that they be CLI-addressable on any env whose scenery is a RealTerrain via Hydra’s dotted-override syntax:

srb agent teleop -e _ground \
  env.scenery.lat_deg=-89.66 env.scenery.lon_deg=0.0 env.scenery.size_m=2000

This form does not currently work — confirmed, not just untested. Investigated while running the first real moon v2 bake (SPICE/Terrain remaining-work Task 6): Hydra’s ConfigStore struct for env.scenery is built from _ground’s declared default (AssetVariant.PROCEDURAL), and Hydra’s struct mode then rejects any override key that isn’t already present in that struct — ConfigAttributeError: Key 'lat_deg' is not in struct. Prefixing + to add a “new key” gets past struct validation but then fails differently (AssetResolver resolves to the base Scenery class rather than RealTerrain, so the required body field is missing). This is a real Hydra/AssetResolver limitation on this override surface, not a naming typo — no fix has landed for it. The working ways to construct a region request today are the pure Python constructor (RealTerrain(body=..., lat_deg=..., lon_deg=..., size_m=...)) and the dedicated srb asset download terrain --lat --lon --size-m / srb asset stack --lat --lon --size CLI subcommands below — both parse and run.

Region identity: the slug

A region request’s cache identity is a deterministic, machine-independent slug used everywhere a manifest site’s name would be:

region_{body}_{lat_deg:+08.4f}_{lon_deg:+09.4f}_{int(round(size_m))}m
# e.g. region_moon_-89.6600_+000.0000_2000m

The request’s fields themselves are normalized before the slug (or anything else) reads them, so two requests within the same quantum are genuinely the same region — same fields, same cache entry, same bake:

lon_deg is wrapped to [-180, 180) (so lon=180 and lon=-180 are one region), and signed zero collapses to 0.0;
lat_deg and lon_deg are rounded to 1e-4 degrees (≈3 m on the Moon) — requests differing only inside that quantum share one cached region;
size_m is rounded to whole metres.

The slug’s field widths are pinned by golden-string tests — changing them would silently orphan every existing cache entry.

Bodies and canonical CRSs

Three bodies are supported end-to-end (Body = Literal["moon", "mars", "earth"]): request path, catalog schema, providers, materials/physics/ lighting/augment presets, and companion/scenery wiring. Each body has a canonical polar pair plus a canonical geographic CRS, resolved by srb.terrain.ingest.planetary_crs.canonical_crs_for and matched against raw DEM headers by normalize_crs (projection-parameter matching first, authority code only as a fast path — see that module’s docstring):

Body	South-polar	North-polar	Geographic
moon	`IAU_2015:30135`	`IAU_2015:30130`	`IAU_2015:30100`
mars	`IAU_2015:49935`	`IAU_2015:49930`	`IAU_2015:49900`
earth	`EPSG:3031`	`EPSG:3413`	`EPSG:4326`

Mars’s polar/geographic entries use the IAU_2015 sphere radius 3396190.0 m (not the 3389500 m best-fit ellipsoid figure sometimes quoted elsewhere). Earth’s polar entries use the WGS84 semi-major axis 6378137.0 m; EPSG:3413’s non-zero central meridian (lon_0 = -45) means it is reachable only via the authority fast path, never by projection- parameter matching (a documented asymmetry with EPSG:3031 — see the code comment at planetary_crs.py’s _CANONICAL_PARAM_SIGNATURES).

Parameter matching is scale-aware as of 2026-08-02: a polar-stereographic grid’s lat_ts/k_0 are normalised to one effective scale at the origin and compared, so an equivalent WKT spelling of EPSG:3031 (standard parallel -71) still matches it, while an unscaled south-polar WGS84 grid — a genuinely different grid, ~2.7% off in ground scale — no longer does. Such a grid now falls through to matched_by="body_default" (EPSG:4326) and keeps its own CRS, instead of being silently treated as EPSG:3031 and skipping the reprojection that would have corrected it. No shipped catalog layer is affected — all of them are unscaled polar or geographic — but a user-supplied local DEM can be.

Body-aware, non-polar-capable resolution

Every request resolves in a body-aware projected CRS (Phase 5b, D12 lift): a polar request (|lat_deg| >= 60°) uses the body’s canonical polar- stereographic CRS from the table above, and any other request uses a per-request local stereographic CRS centered on its own lat/lon — so comparing catalog footprints (also pinned in projected metres) against the request never mixes CRSes. There is no longer a structural non-polar gate: a request with no catalog footprint actually covering it still returns the resolver’s empty plan with a warning, which means procedural fallback in an env and a clear error on the CLI, exactly like the empty-catalog case.

The local CRS (local_stereographic_crs(body, lat_deg, lon_deg)) is a proj string of the form +proj=stere +lat_0=<lat> +lon_0=<lon> +R=<body radius> +units=m +no_defs (Earth uses +ellps=WGS84 in place of +R=), built fresh per request from its own already-quantized (1e-4°) center — it is exact at the center point (the center always projects to (0, 0)) and its distortion over a patch radius up to MAX_REGION_SIZE_M (8 km, the region size cap) is negligible for terrain-scale use; nothing this projection feeds (mesh, footprint containment, location-variety offsets) needs sub-8-km-scale angular accuracy. Because its lat_0/lon_0 are the request’s own center rather than a zero origin, it never matches one of the table’s canonical parameter signatures — normalize_crs on it always reports matched_by="body_default", and the bake’s georef block keeps this proj string verbatim rather than trying to canonicalize it to an authority id. This helper raises at the poles themselves (|lat_deg| >= 90 - 1e-9) — polar requests must go through the canonical-CRS path above instead, since an unscaled oblique-stereographic pole center would silently collide with (and for Earth, differ in scale from) the canonical polar entry.

Footprints: the `bbox:` form

A catalog layer’s footprint field (see Adding Terrain Sites) now accepts a third shorthand alongside cap:<lat><N|S> and WKT POLYGON(...)/MULTIPOLYGON(...) in projected metres:

bbox:<lonmin>,<latmin>,<lonmax>,<latmax>

— a plain geographic-degrees footprint, body-agnostic and independent of whatever projected CRS the region happens to resolve into. This is the form non-polar (and most Mars/Earth) layers are expected to use, since authoring a WKT polygon in a per-region local stereographic CRS that doesn’t exist until request time is impractical. Rules, enforced at schema load and at footprint-containment time:

lonmin/lonmax accept the full [-360, 360] pre-normalization range — the dominant convention for planetary product metadata (LOLA/HRSC/CTX 0–360° east longitude) works unmodified — and are normalized to [-180, 180) once, at parse time, before any containment check runs.
lonmin > lonmax after normalization is a deliberate antimeridian wrap (the box spans ±180°), not an error — containment is lon >= lonmin or lon <= lonmax in that case, mirroring the region-slug’s own lon handling.
latmin <= latmax is enforced (unlike lon, an inverted latitude band has no valid meaning): latmin > latmax raises a pydantic.ValidationError naming the footprint string, at catalog-load time, before any byte moves.
Containment against a request is evaluated by projecting each sample point back to lat/lon (xy_to_latlon in the region’s own resolved CRS, whatever that is — canonical polar or local stereographic) and testing the degree box directly, so bbox: footprints work identically in every body-aware CRS from the section above.

cap: and WKT-in-projected-metres are unchanged; WKT stays usable only when the catalog author and the region resolver agree on a shared canonical projected CRS (effectively: polar layers only, since a non-polar region’s CRS is generated fresh per request and can’t be known ahead of time when the catalog entry is authored).

What happens when a region can’t resolve

An empty DEM plan — any request whose footprint no curated layer covers (most requests today, since coverage is narrow — see Catalog status per body), or a request against a body/region with no catalog coverage at all — raises RegionUnresolvedError inside the bake path. The two consumers treat it differently, on purpose:

Environments (RealTerrain as env.scenery): the existing terrain degrade ladder catches it and falls back to the procedural terrain, with a one-time warning naming the region slug and the reason. Training keeps running; the terrain just isn’t real.
CLI (srb asset download terrain --lat ...): prints the same reason and exits 1 — a download command has no procedural fallback by design: a download that cannot resolve should say so, not silently bake noise. When the body’s loaded catalog really has zero layers, the error is followed by a note naming the catalog file and the dormancy state.

CLI

# Arbitrary region (mutually exclusive with a positional site target,
# --all, and the site-only flags --patch/--patch-size/--center/--no-ortho):
srb asset download terrain --lat -89.66 --lon 0.0 --size-m 2000 \
  [--body moon] [--lod medium] [--deshade] [--force]

# Preview what a region would resolve to (moves no bytes):
srb asset stack --body moon --lat -89.66 --lon 0.0 --size 2000 [--deshade]

# Assess without baking (doctor/info know region-derived cache entries):
srb asset doctor terrain [--deshade]

The region download routes through the exact same bake_or_load path the envs use, so consent, caching, and provenance all apply unchanged. A remediation command printed by doctor/availability (download_command) reproduces the request faithfully — including --deshade when the assessment was deshade-keyed, since a deshaded bake is a different cache entry (below).

Per-env location variety

Region requests (only region requests — a curated site or a local dem_path have a fixed footprint) can fan out into K distinct patches, one baked USD per patch, distributed across environments by the Isaac Lab clone planner instead of every env sharing the same terrain:

from srb.assets.scenery import RealTerrain

# K=8 patches: index 0 is the requested center verbatim; the other 7 are
# deterministic seeded offsets within a 3000 m disk around it.
terrain = RealTerrain(
    body="moon",
    lat_deg=-89.66,
    lon_deg=0.0,
    size_m=2000.0,
    num_locations=8,
    location_spread_m=3000.0,
)

# Explicit centers instead of derived offsets (all share size_m/lod/deshade):
terrain = RealTerrain(
    body="moon",
    size_m=2000.0,
    locations=[(-89.66, 0.0), (-89.40, 15.0), (-89.80, -20.0)],
)

Pre-bake the same set from the CLI (serial, same bake_or_load path, same consent/caching/provenance rules as a single-location download):

srb asset download terrain --lat -89.66 --lon 0.0 --size-m 2000 \
  --num-locations 8 --location-spread-m 3000

Each line printed is <slug>: <path>, one per baked location, in derivation order. --num-locations/--location-spread-m are rejected (exit 2) off the --lat/--lon/--size-m region form, exactly like --patch-size/--center; --num-locations below 1 is also exit 2. A RegionUnresolvedError on any location prints that location’s slug and stops the run — locations after the failing one are never attempted, and any already-baked predecessors are kept (partial completion, not all-or-nothing).

Semantics

Aspect	Behavior
Primary patch	Index 0 is always the requested `(lat_deg, lon_deg)` center verbatim (or `locations[0]` for the explicit-list form). Companions, the SimForge augment layer, and any other single-path reader use `RealTerrain.primary_baked_usd_path`, which anchors to this patch.
Derivation	Requests 1..K-1 come from `numpy.random.default_rng([905, seed])` (a fixed salt decoupling this stream from every other consumer of `seed`), sampled uniformly in a disk of radius `location_spread_m` (`r = location_spread_m · √u`) in the request’s own resolved projected CRS (canonical polar-stereographic near the poles, local stereographic elsewhere — see Bodies and canonical CRSs). `location_spread_m` defaults to `size_m` when omitted.
Prefix-stability	The RNG stream does not depend on `K`, so raising `num_locations` (e.g. 4 → 8) keeps the first 4 derived slugs — and their cache entries — unchanged; it only appends new ones.
Slug uniqueness	After the usual 1e-4 deg / whole-metre normalization, the K requests must resolve to K distinct slugs. A collision (spread too small relative to the quantum, or duplicate entries in an explicit `locations` list) raises `RegionInvalidError` naming the duplicate slugs and suggesting a larger `location_spread_m` or de-duplicating `locations` — never a silent dedupe or re-jitter.
Any latitude	Variety works at any latitude since Phase 5b: the offset math runs in the request’s own projected CRS (canonical polar-stereographic near the poles, local oblique stereographic elsewhere — see body-aware resolution above), so neither derived nor explicit centers are restricted to the polar band anymore.
`locations` vs `num_locations`	Mutually exclusive: `locations` fixes K to its own length and requires `size_m` (it has no single center to pair with `lat_deg`/`lon_deg`); passing both `locations` and `num_locations != 1`, or `locations` together with a `lat_deg`/`lon_deg` center, is a validation error.
Companions	A region request has no curated companions regardless of K (unchanged from the single-location case) — they attach only for a curated `site`.

K==1 (the default) is byte-identical to the pre-Phase-5a single-patch behavior: same UsdFileCfg spawner, same cache entries, same slugs. K>1 instead spawns a genuine MultiAssetSpawnerCfg (func=spawn_multi_asset_compat) at the env-scoped prim path, which Isaac Lab’s clone planner distributes one variant per environment.

Deshade (illumination-corrected colour)

Real sunlit colour mosaics have the sun baked into them: topographic shading from the capture-time illumination fights the renderer’s own lighting. deshade=True corrects for this at fusion time — for each sunlit colour layer, the engine models a hillshade of the fused DEM at that layer’s catalogued capture illumination (sun_azimuth_deg/sun_elevation_deg on the catalog layer) and divides it out, mean-preserving (the layer’s overall brightness does not shift) and NaN-safe (nodata holes pass through untouched). Rules:

PSR colour layers are never deshaded — a permanently shadowed region has no capture-time sun to divide out.
A sunlit layer without catalogued sun angles is left un-deshaded, with a warning naming it that lands in the bake’s meta.json fusion warnings — never a silent skip.
If no fused DEM is available to model hillshade from, colour is left un-deshaded with a warning.

Three knob surfaces, one precedence rule:

Surface	Form
Site manifest	`deshade: true` on the site entry (`srb/terrain/manifest.yaml`)
Python	`RealTerrain(..., deshade=True)`
CLI	`--deshade` on `srb asset download terrain` / `srb asset stack` / `srb asset doctor`

Precedence is manifest or request: a request-level deshade=True can only turn deshading on; it can never disable a manifest-declared deshade: true. The effective flag is folded into the v2 cache key (any knob that changes baked pixels must be), so deshaded and non-deshaded bakes of the same region/site coexist as distinct cache entries; the default False keeps every pre-existing cache key byte-identical.

v1 sites cannot deshade. A v1 (single-source sources.dem) site has no fusion engine in its bake path, so deshade=True there is not silently inert — it raises (DeshadeUnsupportedError), and the CLI checks the same manifest fact before a single byte is fetched rather than after the whole DEM download.

Unchanged from Phase 2: a v2 catalog layer whose estimated fetch exceeds 1 GiB needs both the layer’s curated allow_large_download: true and the operator’s SRB_TERRAIN_LARGE_OK=1 environment variable at fetch time (check_layer_consent, an AND — env-var only, never an interactive prompt). Region requests add no second consent mechanism; the same gate governs every byte the region path moves. See allow_large_download + SRB_TERRAIN_LARGE_OK for the full rules and why srb asset stack’s printed “consent required” line is only a descriptive preview.

Companions and the ephemeris sun

A curated real-terrain site can carry companion products — a sky dome, a lighting rig (possibly ephemeris-baked; see Terrain lighting), and rock sets. As of Phase 4 (OI-1), the env path attaches them automatically: BaseEnvCfg._add_scenery calls attach_companions() whenever the resolved scenery is a RealTerrain with attach_companions_enabled=True (the default; set it False to opt a task out and keep the pre-OI-1 scene).

Curated companion product ledger

Companion lifecycle is product-specific. Source presence and an Isaac-free pipeline contract do not establish publication or rendered-scene readiness.

Product	Family and owner	Current disposition	Publication/runtime acceptance
`apollo17_pan`	Sky dome; `srb/terrain/hdri`	Retain with condition. The curated manifest consumes it, but its catalog URL and hash are placeholders, so it is not publishable.	Supply a licensed stable URL and real hash; verify fetch, texture generation, and `dome.usd`; then capture an Isaac curated-site run attaching `skydome` without fallback.
`jezero_late_morning`	Lighting rig; `srb/terrain/lights`	Retain. This is a procedural static preset consumed by Jezero, not a downloaded O3 product. Runtime acceptance remains deferred.	Verify a `lights.usd` bake and an Isaac Jezero run proving lighting attachment and arbitration without fallback.
`lunar_basalt_set`	Rock set; `srb/terrain/rocks`	Retain with condition. Apollo sites consume it, but both catalog inputs use placeholder URLs and hashes, so it is not publishable.	Supply stable URLs and real hashes; verify downloads plus `identity.json` and `rock.usd` generations; then capture an Isaac run attaching `rocks_00` and `rocks_01`.

Region requests remain outside this ledger because they have no curated manifest entry. SimForge/augment rocks are also excluded: they are synthetic augmentation inputs, not these curated companion products.

Both a companion lighting product and an enabled env.ephemeris want to own “what lights this scene”, so attachment is arbitrated (srb.terrain.companion_scene.arbitrate_companion_scene) under a fixed precedence. Every suppression warns once, naming both conflicting knobs:

#	Condition	Outcome
1	`env.ephemeris.enabled` and `drive_sunlight`	The ephemeris sun owns lighting: the companion lighting product is suppressed (warn-once). If `drive_skydome` is also set, the companion sky is suppressed too. Companion rocks still attach.
2	Else, the companion set contains lighting	The companion owns the scene sun: `scene.sunlight` is set to `None` and the four static sunlight randomizers (`randomize_sunlight_`) are skipped (warn-once when any were configured) — `scene.lighting` and `scene.sunlight` are mutually exclusive. A manifest `lighting: "ephemeris"` site under an env without* `env.ephemeris.enabled` gets its baked ephemeris preset this way.
3	The companion set contains sky	The companion sky replaces the domain-default skydome and `randomize_skydome_orientation` is nulled (deterministic curated sky) — unless the env explicitly configured a non-default skydome string (e.g. `env.skydome=high_res`, which wins over the companion sky) or explicitly disabled the skydome (`env.skydome=null`/`false` — an operator disable also wins; the companion must not silently re-add a dome).

Additional rules:

Region terrains have no curated companions (there is no manifest entry to declare them), so a region request attaches nothing — silently: absence is not a conflict, nothing warns. env.ephemeris is the way to get a real sun on a region patch; a manifest lighting: "ephemeris" is unreachable for region requests until a manifest entry exists.
All-or-nothing degrade. If building/attaching a site’s companions fails — placeholder catalog entries until the O3 data track lands real products, or an offline run — the whole companion set for that site is dropped with a one-time warning and the env builds with the pre-OI-1 scene. Per-product degrade (keep the sky when only the lighting product is broken) is a post-O3 candidate, not implemented.

Rendered-output change, deferred by data. The wiring above changes what a curated-site env renders compared to pre-OI-1 — a site’s curated sky/lighting now actually reaches the running scene. Today that change is latent: the shipped sites’ companion catalog entries are placeholder products (O3-blocked), so attachment degrades with the warn-once and envs render exactly as before. The moment real companion products land, curated-site envs will light and sky differently than they did pre-OI-1. Opt out per task with attach_companions_enabled=False, or override the individual companions (sky=None, lighting=None) on the scenery.

Observed against the first real moon v2 bake (2026-08). Real-Terrain-v2 Task 5 curated srb/terrain/catalog/moon.yaml with 12 real, sha256-pinned DEM/colour/mask source layers, and a real region bake (region_moon_-89.6600_+000.0000_2000m) was run against it end to end: the DEM (windowed COG read) and the sole covering colour product (lroc_wac_emp_643nm_south_100m, a single-band 643 nm reflectance mosaic — 1,394,200,920 bytes, sha256-verified) and mask (pgda_lpsr_85s_60m, 656,183 bytes) all fetched correctly over the real network. This confirms the paragraph above’s premise does not apply to region requests at all, regardless of data: per “Additional rules” above, a region request has no manifest entry and therefore attaches zero companions unconditionally — real colour landing in the per-body source catalog (moon.yaml/ mars.yaml/earth.yaml) can never flip that, by construction. The companion track this note is actually about is the separate presentation catalogs — srb/terrain/hdri/catalog.yaml, srb/terrain/lights/catalog.yaml, srb/terrain/rocks/catalog.yaml — which back curated-site sky=/rocks= (lighting= presets that aren’t "ephemeris" are procedural, not downloaded, so they are unaffected). Those still carry only placeholder file:///tmp/... entries with dummy sha256 hashes; Task 5’s moon.yaml curation is a different data track and does not touch them. So this note stays open for curated sites, now for a more specific, verified reason: it needs its own O3-equivalent curation pass over the hdri/lights/rocks catalogs, not (only) the per-body source catalogs.

Update (SPICE/Terrain remaining-work, Task 15, 2026-08): the bake above now completes end to end. At Task 6 time, real single-band colour data exposed a pre-existing ingest defect: a (H, W, 1) array — srb.terrain.ingest.dem.load_dem_window_multiband’s documented shape for a genuinely 1-band “colour” product — was rejected by srb.terrain.fusion.color._to_hwc3_float64, which only accepts (H, W) or (H, W, 3). Task 15 fixed this (D-C) at the api.py ingest call site — replicating a genuinely single-band colour product to 3 channels before it reaches fusion/, which stays untouched (git diff -- srb/terrain/fusion/ empty throughout) — alongside two sibling defects in the same windowed-read path found by the same investigation: D-A (no CRS reprojection between a source’s native transform and the destination grid) and D-B (non-square geographic pixels rejected outright). The same region (region_moon_-89.6600_+000.0000_2000m) now bakes end to end, area_fraction=1.0 on every base layer, and three more real bakes (Mars Jezero, Earth polar, Earth non-polar) complete the same way. Region requests still attach zero companions regardless (unchanged structural fact from “Additional rules” above), so this still does not exercise companion-arbitration code or its degrade path — that observation stays open until the hdri/lights/ rocks catalogs get their own curation pass, independent of whether the per-body source catalogs or the fusion ingest path work.

Illumination sidecar (SPICE Phase 5)

A baked patch can carry a precomputed illumination sidecar (illumination.npz, schema srb_illum/2) alongside its terrain.usd — a separate, out-of-band artifact produced by srb ephemeris import, not a bake input (it is not part of the CacheKey). RealTerrain.illumination_product_path exposes it (derived from the primary/index-0 patch only — see the per-env location variety limits above for what that means under num_locations > 1), and srb/core/mdp/illumination.py’s masked-spawn term and illumination_fraction_at reward helper consume it. See Illumination Products for the full schema, importer, and consumer reference.

Augmentation database (`srb_augdb/1`, Phase 5c)

A baked patch can carry a deterministic augmentation database — versioned synthetic crater/rock placements derived from the patch’s own fused truth, alongside terrain.usd. It is generated at bake time by srb.terrain.augmentation.database.generate_augmentation_db and persisted as a compressed sidecar (augmentation.npz) via save_augmentation_db.

Schema (srb_augdb/1, AUGMENT_DB_VERSION = 3). Two float32 arrays. x_m/y_m are patch-local mesh-frame meters (x = grid east, y = grid north, origin at the patch center — the same XY frame as mesh_frame {"center_xy": True}):

Array	Shape	Columns
`craters`	`(Nc, 4)`	`x_m, y_m, radius_m, depth_m`
`rocks`	`(Nr, 6)`	`x_m, y_m, z_m, scale, yaw_rad, asset_idx`

z_m (SPICE/Terrain remaining-work Task 10, db_version bumped 1→2) is sampled bilinearly from the bake’s own DEM at generation time, 0.0 when no DEM was available. z_m is on the mesh’s Z datum, not the DEM’s raw datum (re-review finding C1, db_version bumped 2→3): it is the bilinear DEM sample MINUS the same centre-pixel origin heightmap_to_mesh(..., z_origin="center") subtracts from the mesh — i.e. the same mesh_frame {"z_origin": "center"} the baked USD itself uses — so a rock’s (x_m, y_m, z_m) lands directly on the baked mesh surface. db_version == 2 sidecars predate this fix and carry z_m on the DEM’s raw (often kilometres-off-zero for real planetary DEMs) datum instead; there is no in-memory upgrade for that case, only a rebake. load_augmentation_db_file auto-upgrades a pre-existing v1 sidecar in memory (inserts a zero z_m column at index 2, records meta["upgraded_from"] = 1) so old cache entries keep loading without a rebake. Zero-density kinds are (0, 4)/(0, 6) arrays, never None. A meta JSON block records schema, db_version, seed, slug, body, preset, per-kind counts (requested vs. kept, see the shortfall rule below), provenance: "synthetic", and generated_from (patch_size_m, gsd_m, max_slope_deg, and dem_stats — None when no DEM array was available to seed the slope-exclusion proxy).

Generated from fused truth, not from nothing. bake_or_load passes the in-memory fused DEM it already holds at bake time (dem.heights, dem.gsd_m) into the generator; positions are otherwise uniform over the patch, then rejection-resampled away from cells whose local slope-magnitude proxy (np.gradient of the DEM) exceeds max_slope_deg (25° by default). Densities and size ranges come from the body’s AugmentPreset (rock_density_per_m2, rock_size_range_m, crater_density_per_m2, crater_radius_range_m — the crater fields exist in every preset today, but every shipped preset’s crater_density_per_m2 is 0.0, so shipped databases carry rocks only, in (0, 4) crater arrays).

Determinism/version contract. The generator’s RNG stream is np.random.default_rng([761, AUGMENT_DB_VERSION, seed, *sha256(slug)-ints]) — no hash() (PEP 456 randomizes it per process), no dict-iteration-order dependence. The same (seed, db_version, slug) triple — plus the same preset and DEM input — reproduces byte-identical craters/rocks arrays on any machine, in any process, regardless of hash-randomization or unrelated np.random global-state use elsewhere in the process. Changing the stream (the salt, the RNG construction, the placement rules, the schema layout) requires bumping AUGMENT_DB_VERSION — the version is folded into the seed itself, so a bump silently re-derives every existing database rather than colliding with it.

Shortfall on steep terrain. Slope-exclusion never “loops until satisfied” (that would make the stream length, and thus determinism, depend on the acceptance rate): the generator draws a fixed multiple (_OVERSAMPLE = 4) of candidate positions up front and keeps the first N that pass the slope test, in draw order. If fewer than N candidates survive (e.g. a patch that is mostly steep), the database simply carries fewer placements than the density implies — recorded honestly in meta["counts"] (*_requested vs. *_kept), never silently retried to make up the difference.

Bake wiring. Generated for both v1 and v2 bakes alike, but only when the body’s effective augment preset has nonzero rock or crater density — moon and mars presets do (rocks), earth’s EARTH_DEFAULT is all-zero, so earth bakes carry no augmentation.npz and no meta.json block at all. Generation runs inside _bake, writes into the tmp bake directory before the same atomic promote loop that lands provenance.tif, and is registered as meta.json["augmentation_db"] = {"file": "augmentation.npz", "schema": "srb_augdb/1", "db_version": 3, "counts": {...}}. A cache hit never regenerates. Generation failure (including the DEM/gsd sanity guard tripping on a malformed input) degrades with a logged warning — it never fails the whole bake, since the sidecar is a bonus artifact, not a requirement for a usable terrain. The sidecar is not part of CacheKey — like the illumination sidecar, it is derived from an already-baked patch and cannot influence the cache key that produced it.

Read API. srb.terrain.augmentation.database.load_augmentation_db(baked_dir) returns an AugmentationDb | None — None when the meta block or the sidecar file is absent; a file that is PRESENT but fails schema/shape/ finiteness validation raises ValueError (a corrupt sidecar is an error, not a silent absence). RealTerrain.augmentation_db_path mirrors illumination_product_path’s “sidecar next to the primary baked USD” convention, including the same per-env location-variety limit: only the PRIMARY (index-0) patch’s database is exposed when num_locations > 1.

Honesty note (D1, updated by Task 11). Phase 5c shipped generation, persistence, the read API, and a cross-process determinism proof. Bake-time crater stamping is now implemented, opt-in (srb.terrain.augmentation.stamping.stamp_craters): set TerrainSpec.augment_stamp = True (also exposed as RealTerrain.augment_stamp) and the generated database’s craters are cut into the DEM itself — a parabolic bowl to the rim radius R, then a raised-cosine rim collar decaying back to grade at 1.4 R, rim height 0.06 * depth (a documented profile, not a physical simulation) — inside _bake, right after the nodata-repair step and before heightmap_to_mesh runs, so the deformation is baked into the mesh, materials, and collision alike, not a cosmetic overlay; rock z_m is re-sampled on the stamped surface so placements still sit on the ground. augment_stamp = False is the default and is inert everywhere: every shipped preset keeps crater_density_per_m2 = 0.0, so stamping only does anything once paired with a preset that sets it (lunar_cratered/ mars_cratered — dataclass copies of the body defaults with crater_density_per_m2 = 0.002 and crater_radius_range_m = (1.0, 8.0), or a future custom one). Selecting that preset (re-review finding I2) is RealTerrain.augment_preset = "lunar_cratered" (or "mars_cratered"), threaded verbatim into TerrainSpec.augment_preset — _bake resolves spec.augment_preset or default_augment_preset_for_body(spec.body) for BOTH sidecar generation and stamping, so an explicit override actually reaches the bake instead of _bake silently re-resolving the (always crater_density_per_m2 = 0.0) body default underneath an augment_stamp = True that then has nothing to stamp. augment_preset must name a preset whose own body matches the spec’s body; an unknown name or a wrong-body one is rejected loudly, at RealTerrain(...) construction time, not lazily at bake time (re-review finding I4 — corrects an earlier version of this note that claimed the opposite): bake_or_load computes a CacheKey unconditionally before _bake ever runs, and CacheKey.from_spec resolves the effective preset through companion_scene.resolve_effective_augment_preset — the single helper shared by the cache key, _bake’s generation block, and companions()’s SimForge/database augment layer — which raises KeyError for an unknown name and ValueError for a known preset whose body disagrees, so all three consumers of this one field fail the same way, at the same time, naming the offending value and the presets available for that body. Unlike the sidecar itself, this knob does reach CacheKey: hash12() appends augment_stamp={db_version}:{density:g}:{radius_min:g}:{radius_max:g} (the resolved — override-or-body-default — preset’s own crater knobs) whenever augment_stamp is True or an explicit augment_preset was set (the latter changes GENERATED craters/rocks even when stamping is off) — the same append-only “inert default” precedent deshade=1/ synthetic=1 established, so every existing golden hash and every default (no override) bake’s cache key are unaffected. A stamping failure degrades the same way generation does — a logged warning, DEM left unstamped, never a broken bake. Rock placements are now wired into the scene, explicitly, via augment="database" (Task 12 — see below); this closes the augmentation clause’s “identical scatter across machines” half for rocks. Crater placements are consumed by stamping (the DEM deformation above) but are not separately spawned as scene prims — a crater has no standalone mesh to place, only a DEM cut.

Explicit rock scatter — `augment="database"` (Task 12)

RealTerrain.augment gains a third mode alongside "none" (default) and "simforge" (legacy, unchanged): augment="database" places rocks itself, at the exact transforms recorded in the bake-time augmentation database sidecar, instead of delegating to SimForge’s own internal scatter.

Why a third mode. augment="simforge" hands SimForge a rock count and a seed via SimforgeAssetCfg/spawn_simforge_assets — SimForge decides where each rock goes internally, with no per-instance transform API SRB can read back. augment="database" instead reads the (Nr, 6) rocks array already generated by srb.terrain.augmentation.database ([x_m, y_m, z_m, scale, yaw_rad, asset_idx], patch-local mesh frame) and spawns one static prim per row at that row’s own position/scale/yaw — the same positions every time, on every machine, because the sidecar itself is already proven deterministic (see above).
Variant baking. srb.terrain.augmentation.database_scatter bakes a small, fixed catalog of _ROCK_ASSET_SLOTS = 8 rock variant USDs once (via SimForge’s own generator machinery — the same asset.generator_type(...).generate_subprocess(...) call SimForge’s spawner uses internally, just without the scatter half), cached by SimForge’s own subprocess cache. Each database row picks a variant by asset_idx % len(variants), so _ROCK_ASSET_SLOTS need not exactly equal the row count.
Placement. rock_pose(row) maps a row to (pos, quat_xyzw) — pos = (x_m, y_m, z_m) unchanged, and a yaw-only rotation about z: (0, 0, sin(yaw/2), cos(yaw/2)) in xyzw (w-last) order, matching Isaac Lab’s AssetBaseCfg.InitialStateCfg.rot convention in SRB’s fork (identity (0, 0, 0, 1)) — verified against the fork’s isaaclab/assets/asset_base_cfg.py and isaaclab/utils/math.py before writing a single orientation value, since this fork is xyzw, not mainline IsaacLab’s wxyz.
Cap semantics. RealTerrain.augment_max_rocks: int = 256 caps the number of rocks placed. select_rows takes the first N rows in the database’s own draw order — never a random subsample — so which rocks get placed under a cap is itself deterministic and stable across repeated runs with the same sidecar.
Degrade, never crash. A missing sidecar (no augmentation.npz/ meta.json block) or one with zero rocks logs a warning naming the baked directory and yields no rock companions at all — the terrain still spawns.
Earth has no rock generator. simforge_ext._build_rock_generator now raises ValueError for any body other than "moon"/"mars" (a defense-in-depth guard, not a promise of earth rock data): every shipped earth preset ships rock_density_per_m2 = 0.0, so the zero-density short-circuit in build_real_terrain_augment_cfg — and the empty-rocks degrade path above for augment="database" — means this raise is never reached by a default-configured bake.
"simforge" is unchanged. The legacy mode’s placement, seeding, and behavior are untouched by this task; augment="database" is purely additive, and the default (augment="none") is unaffected either way.

Synthetic super-resolution layer slot (Phase 5c)

The catalog schema’s radiometry enum includes a "synthetic" class, reserved for a future super-resolution or model-generated DEM/color product. The resolver treats it as a lowest-priority, opt-in slot:

Skipped by default. radiometry == "synthetic" layers never become candidates at all in _build_sorted_candidates unless the request explicitly admits them — a catalog with one synthetic DEM layer and one real DEM layer resolves to the real one only, no warning needed (there is nothing wrong to warn about).
Strictly last when admitted. The resolver’s total sort key is now (is_synthetic, priority_class, native_gsd_m, layer_id) — a synthetic layer sorts after every non-synthetic candidate regardless of its own priority_class/native_gsd_m. Even a synthetic priority_class=0, native_gsd_m=0.5 layer loses to a real priority_class=9 layer. This key is byte-equivalent to the pre-Phase-5c 3-tuple whenever no candidate is synthetic (which is every shipped catalog today), so default resolution is unaffected.
Declared-stack sites admit synthetic layers implicitly. A v2 site’s manifest-declared dem_stack/color_stack entries are matched by layer_id, not filtered by radiometry — naming a synthetic layer in a manifest is explicit authorial intent, so it is never skipped the way an unrequested footprint-search candidate would be. The opt-in gate only matters for footprint-searched (resolve_stacks) and declared-site mask/quality/PSR-color candidates.

Opt-in surface. RegionRequest.allow_synthetic: bool = False and TerrainSpec.allow_synthetic: bool = False (threaded through exactly like deshade). The flag is deliberately not part of the cache-site slug (a region’s identity string), but it is pixel-changing once a catalog ships a synthetic layer, so CacheKey.hash12() appends "synthetic=1" to the hashed material only when the flag is True — the same append-only “inert default” precedent deshade=1 established. allow_synthetic=False keeps hash12() byte-identical to every existing golden hash; the default plan for every existing catalog is unaffected either way, since none ships a synthetic layer.

A region-form TerrainSpec.allow_synthetic=True whose region sub-spec disagrees (region.allow_synthetic=False) raises at spec-construction time — region terrain specs read the effective flag from RegionRequest.allow_synthetic, so a spec-level flag that the region does not also carry would silently do nothing, which the boundary check refuses to allow.

Scope. The slot is schema + resolver plumbing only (D7/D8). No shipped catalog carries a synthetic layer; no super-resolution model is integrated. This is unrelated to the augmentation database’s rock scatter (now wired explicitly via augment="database", Task 12, above) — no synthetic-radiometry product feeds rock/crater placement today, and nothing here changes with that landing.

Current limits

Catalog coverage is real but narrow. The shipped catalogs carry 19 (moon) / 2 (mars) / 2 (earth) real, sha256-pinned layers (see Catalog status per body) — a region request outside every curated footprint still degrades/errs as described above. access: cog_remote remains a compatibility label, but production first downloads and SHA-256-verifies the complete source through SRB’s bounded, redirect-validating provider. Rasterio windows that local payload. Direct GDAL /vsicurl network access is refused until its transport can enforce a per-hop destination-host policy. First use therefore needs full-source space and explicit large-download consent where applicable; later use reuses the content-addressed source and patch caches. Each contributing layer records bytes_verified: true in meta.json fusion provenance. LDAM is pinned and production-baked, but remains a coarse 1064 nm product behind the 643 nm WAC_EMP base in their overlap; a spectral seam is possible. No shipped Moon DEM covers the 50S–60S annulus where LDAM is the sole colour source, so its isolated-catalog contribution proof does not create new public end-to-end terrain coverage there. The NAC_POLE_SOUTH_CM_065/CM_135 mosaics are no longer blocked — the one band-1 tile covering Connecting Ridge is pinned per epoch as a reference_only: true comparison reference (never bake input); the remaining tiles stay unpinned, a multi-GB-download operator decision per site. SfS A3CLR is no longer among them — it was pinned on 2026-08-02 for the two regions containing sites this repo already bakes (Connecting Ridge, Haworth); the other eleven A3CLR regions remain uncurated, not blocked.
Non-polar coverage exists but is thin. Non-polar resolution itself is no longer gated (Phase 5b, D12 lift — any latitude projects and resolves against the catalog), and the earth non-polar bake (3DEP, local stereographic) proves the path against real data; but only that one non-polar layer is curated today.
Real Mars/Earth product curation is delivered, not exhaustive. The Phase 5 exit criterion’s real-data half is now demonstrated in-repo: one real Mars HiRISE bake (Jezero) and two real Earth bakes (Copernicus polar, 3DEP non-polar) complete end to end (SPICE/Terrain remaining-work Tasks 7/8/15). This is one curated site per body, not broad coverage — most Mars/Earth locations still have no catalog layer.
A colour read carries no validity mask. A cross-CRS windowed read of a source that declares no nodata and is not floating-point (integer imagery) now marks out-of-footprint pixels with a warp alpha band, and the single-band DEM reader turns that into NaN — this subsystem’s nodata currency. The colour reader (read_planned_window_multiband, which returns (H, W, C) in the source’s native integer dtype) has nowhere to put it: Dem has no mask field, and NaN would force the float32 upcast the integer path exists to avoid. So a colour read returns whatever GDAL wrote in those pixels (0). Closing this needs a mask channel across the ingest↔fusion interface; it is deliberately not faked in-band. See srb/terrain/ingest/window.py::_vrt_fill_nodata_for.
No credentialed source access. The three Phase 5b providers (uahirise, copernicus_s3, usgs_3dep, above) only reach public, unauthenticated hosts/buckets — a product behind an API key or a private bucket cannot be curated today.
No UTM (or other non-zero-origin) canonical CRS entries. The authority/parameter-matching scheme in planetary_crs.py assumes a canonical entry’s x_0/y_0/lon_0 are all zero (see the module docstring); a UTM zone’s non-zero central meridian is structurally unrepresentable as a canonical entry the way EPSG:3413 already is not (reachable by authority id only). The per-region local stereographic CRS covers the practical need instead.
A hot region cache hit still pays the resolve cost — the v2 cache key needs the resolved plan’s stack hashes, so a catalog load + footprint resolve runs before the cache can even be checked, on every call.
srb asset stack region previews and region downloads share the resolver but not a cache — the preview is recomputed each run (it is cheap and moves no bytes).
Companion degrade is all-or-nothing per site (above).
The augmentation database is read-only by default — craters are cut into the DEM only when the opt-in TerrainSpec.augment_stamp knob is set (lunar_cratered/mars_cratered presets exist for exactly this; see the honesty note above), and it is still not consumed by SimForge’s scatter path; outside of stamping, the only proven consumption is the read API and the cross-process determinism test. No catalog ships a synthetic layer — the super-resolution slot’s opt-in gate, demotion, and cache-key append are exercised against a fixture catalog only, like the rest of the v2 resolver machinery.
Per-env location variety (num_locations/locations) assigns variants to environments uniform-randomly with replacement — Isaac Lab’s default cloner_strategies.random, not round-robin — via torch.randint with no explicit generator, so the assignment is per-run nondeterministic unless the caller seeds torch’s global RNG; some environments can end up sharing the same patch and others may not draw a given patch at all.
Memory/VRAM scales with K. All K patch prototypes stay resident on the stage (under /World/template) regardless of num_envs — a K=8 request costs roughly 8x the terrain-asset memory of K=1, even in a 2-env run.
K competes for the clone-planner’s fixed combination budget. A region’s MultiAssetSpawnerCfg is an unshrinkable group in multi_asset_compat.convert_scene_multi_variant_spawners — it multiplies the scene’s fixed_product rather than being capped itself — so a larger K leaves less of the fixed MAX_CLONE_PLAN_COMBINATIONS = 65536 budget for other multi-variant spawners in the same scene, shrinking SimForge spawners’ num_assets budgets (rocks, procedural scatter, etc.) to keep the total combination count under the cap.
K bakes run serially and eagerly at RealTerrain construction time — the CLI and RealTerrain.model_post_init both bake location 0, then 1, … K-1 in a single synchronous loop before the cfg is usable; there is no lazy or parallel bake path. Each location still pays the “cache hit still pays a resolve” cost noted above, K times over.

Agent Configuration

The Space Robotics Bench leverages Hydra for managing agent configurations across different robot learning frameworks. This document provides an overview of the configuration structure and common parameters for training agents in SRB.

Default Hyperparameters

The default hyperparameters for all algorithms and environments are available under the space_robotics_bench/hyperparams directory. If you do not wish to use Hydra, you can directly modify these files before training your agent.

Note: The available hyperparameters and their structure are specific to each framework and algorithm.

Modifying Configurations

You can modify agent configurations in several ways:

1. Command-Line Overrides

srb agent <WORKFLOW> --algo <ALGO> --env <ENV> \
    agent.learning_rate=0.0001 \
    agent.batch_size=64 \
    ...

2. Configuration Files

Default agent hyperparameters are stored as configuration files under hyperparams/, organized by framework (YAML for most frameworks; JSON for Robomimic):

hyperparams/
├── sb3/           # Stable Baselines 3 (ppo, sac, td3, ...)
├── sbx/           # SB3 eXtensions
├── skrl/          # SKRL (ppo, a2c, ppo_rnn, ...)
├── rsl_rl/        # RSL RL
├── robomimic/     # Robomimic
├── dreamerv3.yaml # DreamerV3 (single-algorithm framework)
└── tdmpc2.yaml    # TD-MPC2 (single-algorithm framework)

You can modify these files directly or override specific values via the command line. Algorithm names always include their framework prefix (e.g. sb3_ppo, sbx_sac, skrl_ppo_rnn, rsl_rl_ppo, dreamer, tdmpc2):

srb agent train --algo sb3_ppo --env peg_in_hole \
    agent.learning_rate=0.0003 \
    agent.batch_size=128

Note: The available hyperparameters and their structure depend on the chosen framework and algorithm. Refer to the respective framework documentation for details.

Tuned configurations

These files are also the destination for hyperparameters tuned with srb agent tune. Two layers are resolved task-specific → global, in the requested observation modality:

hyperparams/<framework>/<algo>.yaml — global per-algorithm set (state observations); <algo>_visual.yaml for visual observations.
hyperparams/task/<task>/<framework>/<algo>.yaml — optional per-task override that takes precedence for that task only.

Exported files carry a provenance header (# Tuned by srb agent tune …) recording the study, trial, objective, seeds, date, git SHA, and Optuna version. They are adopted via ordinary git review. See the Hyperparameter Tuning workflow.

Event Camera (IMX636)

An event camera is a bio-inspired vision sensor: instead of capturing full frames at a fixed rate, each pixel independently reports brightness changes with microsecond timestamps. This makes it well-suited to the high-dynamic-range and fast-motion conditions common in space scenes (harsh lunar shadows, direct sun, rapid docking). SRB ships a synthetic one so you can generate event-stream datasets entirely in simulation.

SRB provides a synthetic neuromorphic-vision sensor that emits an asynchronous event stream modelled on a Prophesee IMX636 (EVK4), together with live synthetic storage, validation, and export paths. Isaac task-scene materialization and event-observation delivery are implemented and wired, but every concrete provider is one-shot and none ships in the default container image, so this guide does not claim provider-backed rollout fidelity or real EVK4 fidelity.

The sensor is standalone — it owns its own pose, intrinsics, internal high-frame-rate renderer, IMX636 sensor-model preset, and per-env RNG. It is not a Camera subclass; events are not frames.

Current rollout readiness is tracked in the Status Matrix. The local PRD is the event-camera source of truth, and Physical EVK4 Intake documents the future real-capture handoff.

Architectural decisions covered in docs/adr/0002-event-camera-sensor.md. PRD: local event-camera PRD.

Overview

Aspect	Choice
Default backend	`v2e` (canonical)
Default preset	`imx636_nominal`
Default sub-render rate	250 Hz
Frame interpolation	Off (SuperSloMo upsampling disabled by default)
On-disk format	`StorageFormat.EVENT_HDF5` (one file per env-episode-sensor)
Manifest	`events_manifest.jsonl` (one row per env-episode-sensor)
Async writer	Yes (background thread, bounded queue, drop-oldest on saturation)
Concurrent reads	Yes (HDF5 SWMR mode)
Compression	Blosc:zstd-3 (preferred) → gzip-6 fallback

Configuration

Construct an EventCameraCfg to describe the synthetic sensor configuration. BaseEnvCfg materializes it into the active Isaac scene as a native RGB camera under the same logical name, keeps the Isaac-free config in a scene-side binding, and the runtime registers the matching event observations.

from srb.core.sensor.event_camera import EventCameraCfg

cfg = EventCameraCfg(
    prim_path="/World/robot/ee/event_cam",
    resolution=(1280, 720),       # IMX636 native
    sub_render_hz=250.0,          # honored from S3 (#19) onward
    preset_name="imx636_nominal", # see "Presets" below
    backend="v2e",                # canonical default
    representation="voxel_grid",  # see "Representations" below
    rng_seed=None,                # None ⇒ derived from episode × env_id
)

Key fields:

Field	Meaning
`resolution: (W, H)`	Render size before ROI / downsample. IMX636 native is `(1280, 720)`.
`roi_crop: (x, y, w, h)?`	Optional crop applied before backend; affects both observation and on-disk events.
`downsample: int`	Integer downsample factor applied after ROI.
`preset_name: str`	Selects an `IMX636Preset` whose values are copied into the threshold / noise fields. Per-field overrides take precedence.
`sub_render_hz: float`	Internal high-frame-rate renderer (250 Hz default). S3 (#19) honors it.
`representation: str`	Observation tensor builder: `voxel_grid` / `histogram` / `time_surface` / `stacked` / `raw`.
`time_bins: int`	T dimension for grid representations (default 5).
`backend: str`	Registry name of the `EventGenerator` to use.
`rng_seed: int?`	Per-env seed; defaults to `episode × env_id` if unset.

Presets

Presets capture IMX636 photometric + noise parameters. Built-ins ship in srb.core.sensor.event_camera.preset:

Preset	`pos_thres`	`neg_thres`	`sigma_thres`	`cutoff_hz`	`shot_noise_hz`	Use case
`imx636_nominal`	0.20	0.20	0.03	50	5	Generic indoor / orbital lighting.
`imx636_low_light`	0.20	0.20	0.05	20	20	Lunar permanently-shadowed regions, deep night.
`imx636_outdoor_sunlit`	0.20	0.20	0.02	70	2	Direct-sun Mars / lunar daylight.
`clean`	0.20	0.20	0.00	0	0	Fidelity-regression baseline (no noise).

from srb.core.sensor.event_camera.preset import (
    IMX636Preset, register_preset,
)

register_preset(IMX636Preset(
    name="excavation_dust",
    pos_thres=0.18, neg_thres=0.18, sigma_thres=0.04,
    cutoff_hz=40.0, leak_rate_hz=0.15,
    shot_noise_rate_hz=12.0, refractory_period_s=5e-4,
    hot_pixel_frac=2e-4,
))

S15 status

S15 (#32) deliverable so far is the invariant framework, not an absolute recalibration. Tests in tests/unit/test_event_camera_preset.py + tests/unit/test_event_camera_preset_distinguishability.py pin:

The physical-ordering direction of each noise/threshold field between imx636_outdoor_sunlit → imx636_nominal → imx636_low_light. The direction is illumination-physics — a retune cannot silently invert it.
That the three presets produce measurably different event streams under the synth-vs-synth statistical scorecard. A retune that collapses them into one is a regression even if individual envelopes still pass.
That refractory_period_s and hot_pixel_frac stay constant across illumination presets (hardware-driven, not illumination-driven).

Absolute-value calibration against real EVK4 captures is gated on:

S16 (#25) shipping real captures via srb dataset capture-real-evk4.
The SuperSloMo mirror landing per Weight Mirrors so the v2e backend can run end-to-end.

Until both land, retunes should preserve the physical ordering and the distinguishability invariants — those are the contracts other code (and the fidelity scorecards) rely on.

Backends, extras, licenses

Backends register against the EventGenerator interface and are selected by cfg.backend = "<name>". Each non-trivial backend ships as an opt-in extra.

Backend	Module	Extra	License	Status	Notes
`noop`	`srb.core.sensor.event_camera.backend.NoopBackend`	(built-in)	MIT	S1 #15 ✓	Emits zero events. Walking-skeleton smoke.
`v2e`	`srb.core.sensor.event_camera.backends.v2e.V2eBackend`	`srb[event-v2e]`	MIT (v2ecore)	S2 #16 (live)	Canonical default. SuperSloMo upsampling + IMX636 noise model.
`metavision`	`srb.core.sensor.event_camera.backends.metavision.MetavisionBackend`	`srb[event-metavision]`	Apache-2.0 (openeb)	S7 #21 (live, integration unverified)	Wraps openeb’s `GPUEventSimulator`. `supports_batching=True` — the recorder constructs one shared instance per sensor and calls `process_batch` once per sub-tick (see S9 below). openEB Python import path is pinned to `metavision_core_ml.event_simulator.GPUEventSimulator` and locked by `tests/unit/test_event_camera_metavision_backend.py`; a future PR with openEB installed in CI validates the path against a live install.
`v2ce`	`srb.core.sensor.event_camera.backends.v2ce.V2CEBackend`	`srb[event-v2ce]`	MIT	S8 #22 (live, opt-in smoke)	Learned event simulator (torch). `supports_batching=True` — recorder routes through `process_batch` once per sub-tick (see S9 below). The adapter supports both a package-style `v2ce.inference.V2CE` provider and the script-style `ucsd-hdsi-dvs/V2CE-Toolbox` layout (`scripts.v2ce_3d` + `scripts.LDATI`). Checkpoint is not shipped — fetch the private `hf-andrejorsula-v2ce` mirror and pass `model_path=<path>` at construction; calling `process_frame` without one raises a clear error.

S9 status (env-runtime batched dispatch)

EventBackendDispatch (constructed by EventRecorder) checks factory.supports_batching at construction. For supports_batching=False backends (noop, v2e) it keeps the per-env shape — one EventGenerator instance per (env_id, sensor) slot in _per_env, looped once per env per sub-tick. For supports_batching=True backends (metavision, v2ce) it constructs one shared adapter instance per sensor in _shared, calls process_batch(rgb_batch, t_ns) once per sub-tick, and routes each per-env event tensor back into its per-env buffer. The shared adapter is still responsible for per-env temporal state; Metavision and V2CE lazily allocate one simulator/model slot per env inside that adapter. Episode boundaries call reset_env(env_id, seed=…, episode_id=…) on the shared backend so adapters can zero only the affected slice. Batched backends MUST override reset_env (the dispatch raises ValueError otherwise); only per-env (supports_batching=False) backends fall back to the base-class default, which forwards to reset() and globally reseeds. Per-env buffers, manifest rows, and HDF5 file layout are identical across both paths — only the dispatch shape changes.

AGPL forbidden. AGPL-licensed event repositories (notably ev-ultralytics) MUST NOT appear in any SRB extra or be vendored into the tree. See docs/adr/0002-event-camera-sensor.md §4.

Installing an extra:

pip install 'srb[event-v2e]'

If you select a backend whose extra is absent, the sensor raises MissingExtraError with the exact pip install invocation.

S19 status

S19 (#33) now has two release-smoke layers. tests/unit/test_release_smoke_extras.py is the always-on Phase-A invariant: every require_extra("X") call site under srb/ must have a matching X entry in [project.optional-dependencies]. Without the gate, a documented pip install srb[event-reconstruction] can silently fail with pip: warning: no such extra, and MissingExtraError then advertises an install hint that pip cannot satisfy.

tests/integration/test_event_camera_phase_b_smoke.py is the opt-in Phase-B route. It runs only with SRB_EVENT_CAMERA_PHASE_B_SMOKE=1; set SRB_EVENT_CAMERA_FETCH=1 to fetch from the registered private HF mirrors, or point it at local files with SRB_EVENT_CAMERA_FIXTURE_H5 and SRB_EVENT_CAMERA_E2VID_CHECKPOINT. The smoke validates the private hf-andrejorsula-active-marker-gen4 fixture as EVENT_HDF5, verifies the private hf-andrejorsula-e2vid and hf-andrejorsula-v2ce checkpoint hashes, and executes tiny E2VID reconstruction and V2CE backend forwards through the optional providers.

This is still not the downstream benchmark. The private active_marker mirror is enough for fixture fetch/import/schema and E2VID provider smoke, but the downstream-eval probe still needs a labelled/task-specific Gen4 target, which can come from the physical EVK4 dataset you collect later.

Representations

The observation builder turns events into a fixed-shape tensor consumable by an RL policy. Selected via cfg.representation:

Representation	Output shape	Description
`voxel_grid`	`(B, T, H, W)` float32	Polarity-signed voxel grid over `time_bins` bins (default).
`histogram`	`(B, 2, H, W)` float32	Polarity-separated event counts.
`time_surface`	`(B, 2, H, W)` float32	Exp-decayed most-recent-event-age per polarity.
`stacked`	`(B, 2T, H, W)` float32	Per-polarity voxel grid stacked along channel.
`raw`	`(B, N, 4)` int64 + `(B,) int64` counts	Padded raw event list (length `raw_max_events`).

B is the number of envs; T = cfg.time_bins; H/W are the effective resolution (after ROI + downsample). See S4 (#17) for the implementation.

Render-rate guidance

cfg.sub_render_hz is decoupled from the env step rate. Recommended floors:

Task class	Recommended `sub_render_hz`
Static / slow-manipulation (e.g. inspection)	100 Hz
Mobile robotics (rover traverse)	250 Hz (default)
Excavation, drilling, scoop impact	500 Hz
Landing touchdown, docking contact	1000 Hz

By default v2e runs with SuperSloMo upsampling disabled (use_interp=False). The cfg.frame_interp field is currently not wired into the v2e backend (make_backend does not forward it), so setting it has no effect; enabling SuperSloMo requires constructing the backend with use_interp=True and a SloMo model path.

Recording during rollouts

Every srb agent subcommand (zero, rand, teleop, ros, train, eval, collect) accepts --record-events. The option, provenance fields, scene materialization, observation delivery, and recorder/storage plumbing are all wired: BaseEnvCfg materializes the EventCameraCfg into the task scene, DirectEnv binds it, and the recorder publishes per env-episode-sensor episodes. What is not established is provider-backed fidelity — no concrete backend (v2e, Metavision, V2CE) ships in the default container image, and the only end-to-end Isaac evidence is a focused local runner using a strict test backend. No event-camera fidelity differential pass is claimed.

The command shape is therefore live but provider-gated:

srb agent eval --env landing --algo skrl_ppo --record-events

For the available synthetic (non-Isaac) path, srb dataset record-events remains live. Its EVENT_HDF5 outputs can use the live storage, validation, and export tooling described below.

Dataset layout

The live synthetic srb dataset record-events path produces a timestamped output directory under record_dir:

20260521T140312/
├── events_manifest.jsonl
├── events_env0_ep0/
│   └── events_event_cam_env0_ep0.h5
├── events_env0_ep1/
│   └── events_event_cam_env0_ep1.h5
├── events_env1_ep0/
│   └── events_event_cam_env1_ep0.h5
├── events_env1_ep1/
│   └── events_event_cam_env1_ep1.h5
└── ...

Each manifest event_hdf5_path is relative to this timestamped run directory and points into the nested per-environment episode publication directory. rgb_mp4_path remains a flat filename relative to the same run directory.

The Isaac srb agent --record-events surface writes the same layout once a concrete backend is installed. The layout above does not prove paired RGB output from an Isaac rollout; an rgb_mp4_path is retained as an optional provenance field for the video recorder.

HDF5 layout

/events/t    int64    ns           (N,)   chunked, compressed
/events/x    uint16                (N,)
/events/y    uint16                (N,)
/events/p    int8     ±1            (N,)
/metadata    group    attrs:
                       schema_version, sensor_preset, resolution,
                       sub_render_hz, backend_name, backend_version,
                       backend_commit_sha, representation, time_bins,
                       episode_id, policy_id, policy_checkpoint_hash,
                       task_id, seed, srb_git_sha, isaaclab_version,
                       rgb_mp4_path (optional), success_flag, reward_summary,
                       compression, event_count, dropped_batches,
                       hot_pixel_mask, hf_repo_ids   (JSON-encoded;
                       ADR-0002 Amendment 1)

Schema version is 1 (frozen for v1). Breaking changes bump EVENT_HDF5_VERSION and trigger a converter + republish.

Manifest JSONL

One row per (env_id, episode_id, sensor_name):

{
  "sensor_name": "event_cam",
  "env_id": 0,
  "episode_id": 0,
  "task_id": "landing",
  "seed": 42,
  "event_hdf5_path": "events_env0_ep0/events_event_cam_env0_ep0.h5",
  "rgb_mp4_path": "",
  "event_count": 184321,
  "success_flag": 1,
  "reward_summary": "{\"total\": 2.5}",
  "srb_git_sha": "deadbeef",
  "isaaclab_version": "2.x.y",
  "backend_name": "v2e",
  "backend_version": "1.5.0",
  "backend_commit_sha": "",
  "sensor_preset": "imx636_nominal",
  "resolution": [1280, 720],
  "representation": "voxel_grid",
  "time_bins": 5,
  "sub_render_hz": 250.0,
  "policy_id": "ppo_landing_v3",
  "policy_checkpoint_hash": "abc123…"
}

The manifest row and the HDF5 /metadata group share a single source-of-truth struct (EventEpisodeProvenance); manifest rows additionally carry the cross-episode routing fields (sensor_name, env_id, event_hdf5_path, event_count). Both also carry the ADR-0002 Amendment 1 provenance fields hot_pixel_mask and hf_repo_ids (JSON-encoded in the HDF5 attrs).

The provenance fields (srb_git_sha, isaaclab_version, backend_version, policy_checkpoint_hash, and seed) make a configured synthetic run auditable and provide inputs for a future replay. They are not, by themselves, a deterministic-replay proof, and they do not establish Isaac task delivery or real-data fidelity.

Reading events

For tests and small analyses:

from srb.core.sensor.event_camera import read_event_hdf5

ev = read_event_hdf5("events_env0_ep0/events_event_cam_env0_ep0.h5")
ev["t"], ev["x"], ev["y"], ev["p"]      # numpy arrays
ev["metadata"]                           # dict of attrs

For streaming over large datasets, open with h5py directly; the file is opened SWMR-aware so you can attach while the writer is still active.

Runtime producer bounds

event_chunk_rows bounds every chunk crossing the backend/dispatch seam; observation_window_events separately bounds events retained for the current online observation. With a writer sink, dispatch hands off one validated chunk synchronously before pulling the next, so sink backpressure reaches a genuine provider iterator without accumulating a dispatch result mapping. Batched iterators must yield exactly one entry per environment. Provider iterator, shape, cardinality, sink, or size failures fail the affected environment and abort its staged writer.

Streaming yields must make progress. A per-env iterator cannot yield an empty chunk; a batched yield may contain idle environments but must contain events for at least one active environment. A provider represents an empty interval by ending that iterator. Otherwise dispatch raises event_camera.stream_no_progress, preventing an empty-yield loop from hanging outside every row and writer bound.

strict_streaming=True rejects a backend before construction unless it both declares streaming support and overrides the iterator dispatch will call (iter_frame for per-env backends, iter_batch for batched backends). The current v2e, Metavision, and V2CE upstream APIs return complete event arrays/lists, so their adapters deliberately remain non-streaming. Splitting those results after materialization would bound SRB retention, not upstream allocation, and is not accepted as strict streaming. A future capability flip requires a provider iterator/callback that directly yields bounded, chronologically ordered chunks without first creating the complete result.

Async writer

By default EventHDF5Writer writes asynchronously on a daemon thread to keep the env step non-blocking:

Bounded queue.Queue(maxsize=256) between the env thread and the writer.
On saturation: oldest batch dropped, dropped_batches incremented, a WARNING logged. The final drop count is persisted under /metadata/dropped_batches.
close() enqueues a sentinel and joins the thread, draining everything still queued.

Force synchronous mode (tests, deterministic CI):

from srb.core.sensor.event_camera import EventHDF5Writer

with EventHDF5Writer(path, metadata, async_writes=False) as w:
    w.append(events)

Validation

Three on-demand validation modules (run via CLI, not in the inner loop):

Slice	Command	What it checks
S12 #23	`srb dataset validate-events stats`	Event rate, polarity ratio, ISI distribution vs. real Gen4.
S13 #24	`srb dataset validate-events reconstruction`	Reconstructed frame SSIM (and optional LPIPS) vs. source RGB.
S14 #30	`srb dataset validate-events downstream`	Train flow / detection on synthetic, evaluate on real Gen4.

Validation runs against explicit EVENT_HDF5 inputs. Reference captures are fetched by URL or registered fixture mirror; SRB does not commit capture data to git.

Reconstruction-fidelity (S13):

srb dataset validate-events reconstruction \
  --events events.h5 \
  --rgb-npy frames.npy \
  --times-npy frame_times_ns.npy \
  --output recon_card.json \
  --operator baseline      # 'baseline' (pure-numpy) or 'e2vid' (gated)
  # --lpips                  # optional, requires srb[event-reconstruction]

The baseline operator bins events between successive RGB frame timestamps and min-max normalises the per-pixel polarity sum — pure numpy, always available. The e2vid operator is gated on srb[event-reconstruction]. A private E2VID checkpoint mirror is registered; fetch it with srb event-camera fetch-weights --mirror hf-andrejorsula-e2vid before using the operator. SRB uses the MIT e2vid provider from uzh-rpg/e2calib and does not vendor GPL rpg_e2vid code.

Downstream-fidelity (S14):

srb dataset validate-events downstream \
  --synth-events events.h5 \
  --real-events real_gen4.h5 \
  --task event_flow         # or 'detection'
  --output downstream_card.json
  # --synthetic-smoke       # no real data; adapter-contract smoke only
  # --write-label-template  # create a valid sidecar skeleton for labelling
  # --check-real-target     # validate physical EVK4 labels before harness use

S14 ships as a protocol stub today: the default CLI prints the requirements for a paper-ready downstream-fidelity probe (model family, metrics, gen-gap reporting convention) and writes a schema-stable DownstreamScoreCard with protocol_status="stub". --synthetic-smoke is the no-real-data adapter path: it validates the synthetic EVENT_HDF5 and writes protocol_status="synthetic_smoke" with synthetic contract facts only, not fidelity metrics. The actual train-on-synth + eval-on-real harness lands once a labelled/task-specific real Gen4 target and model adapter are available — the JSON shape will be the same so consumers do not need to rewrite their parsing logic.

The strict real-target contract is live and can be run before the model harness exists:

--write-label-template creates the expected sidecar beside --real-events. The generated file is a valid contract skeleton, not a finished annotation.
event_flow: real_gen4.h5 must be canonical EVENT_HDF5 and have an adjacent real_gen4.flow.npz with flow_xy shaped (T,H,W,2), valid_mask shaped (T,H,W), and integer timestamps_ns shaped (T,).
detection: real_gen4.h5 must be canonical EVENT_HDF5 and have an adjacent real_gen4.detections.jsonl; each record has integer timestamp_ns, boxes_xyxy as 4-coordinate boxes, and matching class_ids.

SuperSloMo weights

v2e’s temporal upsampler needs a SuperSloMo checkpoint. SRB does not redistribute this checkpoint in-repo. The registered Hugging Face mirror is private until release; collaborators need HF repo access plus HF_TOKEN or hf auth login. Two paths exist today:

# Recommended for collaborators with HF access:
srb event-camera list-mirrors                       # show registered mirrors
srb event-camera fetch-weights --mirror hf-andrejorsula-supersloMo

# Fallback for other private mirrors or one-off downloads:
srb event-camera fetch-weights \
  --url https://example.com/SuperSloMo39.ckpt \
  --sha256 <hex digest> \
  --name SuperSloMo39.ckpt

The fetcher downloads to ${XDG_CACHE_HOME}/srb/event_weights/ and verifies the downloaded bytes against the supplied SHA256. Mismatches abort with a non-zero exit so corrupted / substituted weights can never silently produce events.

Private mirrors registered. The default artifact is still a guarded placeholder; srb event-camera fetch-weights with no --mirror / --url refuses to run. See Weight mirrors for the mirror names, hashes, and publishing contract.

Publishing datasets

Two complementary bundlers wrap an EVENT_HDF5 run directory into a publish-ready layout:

# LeRobot v3 (HuggingFace Hub / behaviour-cloning pipelines) — S17 / #26
srb dataset export-lerobot-v3 <event_hdf5_dir> --output ./bundles/lerobot

# Zenodo archival deposit — S18 / #27
srb dataset export-zenodo-bundle <event_hdf5_dir> --output ./bundles/zenodo.tar.gz

The LeRobot v3 bundle materialises an SRB-extended event_camera modality because LeRobot has no canonical event-camera schema. The on-disk layout:

bundle/
├── meta/
│   ├── info.json            # codebase_version v3.0 + srb_event_camera schema
│   ├── stats.json
│   ├── tasks.parquet
│   └── episodes/chunk-000/file-000.parquet  # one row per (env, episode, sensor)
├── data/chunk-000/file-000.parquet          # one row per episode, event pointers
└── events/<sensor_name>/chunk-000/episode-{n:06d}.h5  # canonical EVENT_HDF5

info.json adds an event_path template alongside LeRobot’s standard data_path / video_path, plus an srb_event_camera block with the schema version, registered sensors, presets, backends, and resolutions present in the bundle. The episode parquet carries the provenance fields from the source manifest (srb_git_sha, policy_checkpoint_hash, backend_version, seed, sub_render_hz, etc.) so downstream tooling can filter without re-reading HDF5 metadata; these fields are not a deterministic-replay proof.

fps is set to the max sub_render_hz across rows — for mixed-task bundles (e.g. landing 1000 Hz + traversal 250 Hz) the dataset’s declared temporal resolution matches the highest rate present.

RAW EVT3 import

Existing Prophesee RAW EVT3 recordings can be normalized into the canonical EVENT_HDF5 schema without Metavision/OpenEB:

srb dataset import-raw-evt3 active_marker.raw \
  --output active_marker.h5 \
  --task active_marker \
  --policy-id prophesee_source

The importer decodes EVT3 address and vector CD events, unwraps the 24-bit EVT3 timestamp, writes nanosecond t values, and preserves the source resolution in /metadata/resolution. Trigger, marker, and continued words are ignored because they are not CD events.

Forward compatibility

Real EVK4 capture (S16 #25) remains gated. The future hardware path is planned to target the same StorageFormat.EVENT_HDF5 schema so downstream consumers can share one layout, but no real capture or hardware fidelity is available today.

A CLI stub reserves the on-disk layout today:

srb dataset capture-real-evk4 \
  --task landing \
  --policy /path/to/policy.ckpt \
  --episodes 8 \
  --output ./captures/evk4 \
  --preset imx636_nominal \
  --resolution 1280x720

The stub prints a 6-step trajectory-replay protocol (mount EVK4, calibrate intrinsics against the synthetic preset, configure Metavision streaming, replay policy deterministically, append manifest rows, validate with validate-events stats) and writes a skeleton events_manifest.jsonl under a timestamped subdirectory. Real hardware capture lands when the Metavision driver is wired; the layout it lands into is the one this stub reserves.

References

ADR: docs/adr/0002-event-camera-sensor.md (in the repository, not in this book)
PRD: local event-camera PRD
Module: srb/core/sensor/event_camera/
Tests: tests/unit/test_event_camera_*.py
v2e — Hu et al., “v2e: From Video Frames to Realistic DVS Events”.
Prophesee IMX636 / EVK4 product page and datasheet.

Event-Camera PRD

This page is the local source of truth for the synthetic IMX636 / EVK4 event-camera sensor and dataset pipeline.

Product Goal

SRB should let users record space-robotics policies with a neuromorphic equivalent of the Prophesee IMX636 / EVK4 sensor, validate synthetic fidelity, and publish event-camera datasets for robot-learning, machine-learning, and computer-vision workflows.

Scope

Standalone EventCameraCfg with task-mounted poses, IMX636 presets, backend choice, representation choice, sub-render rate, and per-env RNG.
Pluggable event-generator backends: noop, v2e, metavision, and v2ce.
Canonical EVENT_HDF5 storage plus manifest reproducibility metadata.
On-demand validation: statistical fidelity, reconstruction fidelity, and downstream train-on-synth / eval-on-real fidelity.
Dataset interchange: LeRobot v3, Zenodo bundle, Prophesee RAW EVT3, and ROS bag export/import paths.
Private staging mirrors for weights and fixtures until redistribution terms and release readiness are confirmed.

Milestones

Milestone	Done when	Current state
M1 walking skeleton	Event-camera config, storage, manifest, and docs exist end to end.	Live
M2 training surface	Event observations can be wired into SRB tasks without changing downstream consumers.	Wired end to end (config, scene materialization, observation delivery, recorder); provider fidelity unproven
M3 dataset validation/release	Fidelity validators, interchange formats, and hosted artifacts are ready for publication.	Partly live; public release gated

M2 is wired end to end: standalone EventCameraCfg, synthetic event generation, storage, validation, and export paths are live, BaseEnvCfg materializes the config into the Isaac task scene, and the runtime delivers event_<name> / event_<name>_count observations. What remains open is provider fidelity — every concrete backend materializes complete results, none ships in the default container image, and the only end-to-end Isaac evidence is a focused local runner with a strict test backend. No simulator-backed event fidelity differential pass is claimed.

Current Blockers

Blocker	Why it matters	What is possible without it
Labelled physical EVK4 task dataset	Required for downstream generalization numbers.	Validate schema, write label templates, run synthetic-only harness plumbing.
Public redistribution approval for mirrors	Required before private HF artifacts become public.	Keep private mirrors registered for permissioned collaborators.
Heavy optional-extra CI image	Required for routine OpenEB / E2VID / ROS bag smoke.	Keep opt-in private smoke and lightweight parser/contract tests.
Task model adapter	Required for real event-flow or detection train/eval.	Freeze scorecard, label contract, and adapter-facing CLI shape.
Concrete provider in the release image	Required for a provider-backed event differential in the SRB task runtime; the materialization and delivery seams already exist.	Run the wired path with the strict test backend, plus the live synthetic, storage, validation, and export paths.

User Stories

As a dataset creator, I can record synthetic event-camera runs with enough provenance to reproduce policy, seed, backend, preset, and task context.
As a validation owner, I can compare synthetic and physical EVK4 data without running fidelity checks inside every policy-training loop.
As a downstream researcher, I can train event-flow or detection models on SRB synthetic data and later evaluate on labelled physical EVK4 captures using the same scorecard schema.
As a collaborator, I can fetch private HF mirrors after being granted access, while public release remains blocked until licensing and curation are ready.

References

Status: Event-Camera Status Matrix
Intake runbook: Physical EVK4 Intake
Architecture: docs/adr/0002-event-camera-sensor.md (in the repository, not in this book)

Event-Camera Status Matrix

Canonical readiness view for the synthetic IMX636 / EVK4 event-camera pipeline. Status terms on this page are the source of truth for the overview, mirror contracts, and dataset CLI reference.

Area	Status	Current contract	Gate to next state
Sensor config	Live (scoped)	`BaseEnvCfg._add_sensors` materializes each `EventCameraCfg` into the active Isaac scene as a native RGB `CameraCfg` under the same logical name and keeps the original Isaac-free config in the scene-side event-camera bindings; `DirectEnv` binds those sensors after the scene is built.	A provider-backed fidelity differential with a concrete backend; the scoped local Isaac run used a strict test backend, not v2e, Metavision, or V2CE.
Task adoption	Live (scoped)	Opt-in task toggles retain `EventCameraCfg` and active-sensor intent; the runtime observation assembly discovers each bound sensor and registers its event representation plus event-count observations, merged into the flat observation every step.	A provider-backed delivery differential on a concrete backend, plus long/high-rate rollout evidence, before task adoption is called live beyond the scoped runner.
`noop` backend	Live	Built-in smoke adapter emits zero events.	None.
`v2e` backend	Live (non-streaming)	Canonical default backend; install with `srb[event-v2e]`. Upstream returns complete event arrays, so the adapter declares no streaming support and `strict_streaming=True` rejects it before construction.	Live optional-extra smoke with the private SuperSloMo mirror; public mirror when release-ready.
`metavision` backend	Live (non-streaming)	Optional adapter and import contract are pinned; the provider materializes complete results, so it is rejected under `strict_streaming=True`.	Live optional-extra CI with OpenEB installed.
`v2ce` backend	Live (non-streaming)	Optional adapter supports package-style V2CE providers and the script-style V2CE-Toolbox layout; it also materializes complete results and is rejected under `strict_streaming=True`.	Promote the opt-in V2CE checkpoint smoke into routine heavy CI; public mirror when release-ready.
Synthetic recording	Live	`srb dataset record-events` writes canonical EVENT_HDF5 episodes and `events_manifest.jsonl`. The `srb agent <verb> --record-events` surface materializes, binds, and records the sensor, but no concrete provider ships in the default container image, so Isaac rollout recording is not proven end to end with v2e, Metavision, or V2CE.	Install a concrete provider in the release image and retain a rollout differential; paired RGB MP4 output remains an optional provenance field, and no provider fidelity pass is claimed.
Statistical validation	Live	`srb dataset validate-events stats` compares rate, polarity, ISI, hot-pixel, and sparsity metrics.	Reference Gen4 fixture for routine CI.
Reconstruction validation	Live	`srb dataset validate-events reconstruction` provides baseline SSIM scoring and an E2VID-gated adapter path.	Promote the opt-in E2VID checkpoint smoke into routine heavy CI.
LeRobot export	Live	`srb dataset export-lerobot-v3` wraps EVENT_HDF5 runs with an `event_camera` modality.	Publication smoke against a hosted dataset.
Zenodo / RAW / ROS interop	Live	`export-zenodo-bundle`, `export-raw-evt3`, `import-raw-evt3`, and `export-rosbag` preserve the canonical event run layout.	Heavy optional-extra smoke for RAW and ROS bag writers plus a licensed fixture mirror.
SuperSloMo weights	Private mirror	`hf-andrejorsula-supersloMo` is registered; fetching requires HF repo permission plus `HF_TOKEN` or `hf auth login`.	Make the HF repo public when release-ready.
E2VID weights	Private mirror	`hf-andrejorsula-e2vid` is registered; fetching requires HF repo permission plus `HF_TOKEN` or `hf auth login`.	Make the HF repo public when release-ready.
Gen4 reference fixture	Private mirror	`hf-andrejorsula-active-marker-gen4` is registered; fetching requires HF dataset permission plus `HF_TOKEN` or `hf auth login`.	Confirm Prophesee redistribution terms before public release; downstream harness still needs a labelled/task-specific fixture.
Preset retune	Phase A	Physical-ordering and distinguishability invariants are tested.	Phase B calibration needs `srb dataset capture-real-evk4` data plus SuperSloMo weights.
Release smoke	Phase B opt-in	Optional-extra names are tested without heavy dependencies; `tests/integration/test_event_camera_phase_b_smoke.py` validates the private active-marker fixture plus E2VID and V2CE checkpoints when explicitly enabled.	Promote to routine CI after public/licensed mirrors and a heavy release image are ready.
Downstream validation	Gated (CLI rejects)	Only `--synthetic-smoke` runs. Every other `srb dataset validate-events downstream` invocation, including `--write-label-template` and `--check-real-target`, is refused by the CLI preflight before any output is written because no validated train-on-synth / eval-on-real harness or model adapter exists; the scorecard, label-template, and real-target helpers exist in-tree and are unit-tested.	Land the real harness with a labelled/task-specific Gen4 target and model adapter, then open the preflight gate.
EVK4 capture	Gated (CLI rejects)	`srb dataset capture-real-evk4` parses its arguments and then exits through the CLI preflight before creating any directory, manifest, or protocol output: no validated hardware capture backend and device contract exists in this build.	Requires a validated capture backend, hardware capture campaign, and publication approval.

Status Terms

Term	Meaning
Live	The interface is implemented and usable from SRB today, subject to any documented optional extra.
Live (scoped)	The path is implemented and wired, and a focused local Isaac run exercised it with a strict test backend. It is not a provider-fidelity, long-run, real-data, or hardware claim.
Live (non-streaming)	The backend adapter works, but the upstream provider materializes complete event results, so `strict_streaming=True` rejects it.
Gated (CLI rejects)	The command parses, then exits through a preflight capability gate before producing output.
Placeholder	The command or registry exists, but the default artifact intentionally refuses to download because SRB does not redistribute the data.
Gated	The interface is documented so downstream work can target it, but meaningful use waits on an explicit external artifact, hardware campaign, or harness.
Private mirror	A stable mirror is registered, but the HF repo is private and requires collaborator access plus an HF token.
Phase A	Lightweight invariant or contract is landed.
Phase B opt-in	Data-backed smoke exists in-tree, but runs only when explicitly enabled with local credentials/artifacts.
Phase B	End-to-end data-backed validation still waits on the task-specific artifacts or heavy CI image named in the row.

User Paths

Goal	Command	Status
List registered event-generator backends	`srb event-camera backends --detailed`	Live
List registered checkpoint mirrors	`srb event-camera list-mirrors`	Live
Fetch checkpoint by explicit URL	`srb event-camera fetch-weights --url <url> --sha256 <sha>`	Live
Fetch checkpoint by mirror	`srb event-camera fetch-weights --mirror <short_name>`	Private mirror; requires HF token
List registered fixture mirrors	`srb event-camera list-fixtures`	Live
Fetch fixture by explicit URL	`srb event-camera fetch-fixture --url <url> --sha256 <sha>`	Live
Fetch fixture by mirror	`srb event-camera fetch-fixture --mirror <short_name>`	Private mirror; requires HF token
Record synthetic event-camera data	`srb dataset record-events --output OUT`	Live
Run statistical fidelity validation	`srb dataset validate-events stats --synth SYN --real REAL --output OUT`	Live
Run reconstruction fidelity validation	`srb dataset validate-events reconstruction --events EVENTS --rgb-npy RGB --times-npy TIMES --output OUT`	Live
Run downstream fidelity validation	`srb dataset validate-events downstream --synth-events SYN --real-events REAL --output OUT`	Rejected by preflight; no harness
Run synthetic downstream adapter smoke	`srb dataset validate-events downstream --synth-events SYN --task event_flow --output OUT --synthetic-smoke`	Synthetic contract smoke live; no real metrics
Check downstream real-target labels	`srb dataset validate-events downstream --synth-events SYN --real-events REAL --task event_flow --output OUT --check-real-target`	Rejected by preflight; contract helper exists in-tree
Write downstream label template	`srb dataset validate-events downstream --synth-events SYN --real-events REAL --task event_flow --output OUT --write-label-template`	Rejected by preflight; contract helper exists in-tree
Export to LeRobot v3	`srb dataset export-lerobot-v3 EVENT_RUN --output OUT`	Live
Build a Zenodo bundle	`srb dataset export-zenodo-bundle EVENT_RUN --output OUT`	Live
Export Prophesee RAW EVT3	`srb dataset export-raw-evt3 EVENTS_H5 --output OUT`	Live
Import Prophesee RAW EVT3	`srb dataset import-raw-evt3 EVENTS_RAW --output OUT_H5`	Live
Export ROS bag	`srb dataset export-rosbag EVENTS_H5 --output OUT`	Live
Capture real EVK4 data	`srb dataset capture-real-evk4 --task TASK --policy POLICY --output OUT`	Rejected by preflight; no capture backend

The opt-in Phase-B smoke is:

SRB_EVENT_CAMERA_PHASE_B_SMOKE=1 \
SRB_EVENT_CAMERA_FETCH=1 \
pytest -q tests/integration/test_event_camera_phase_b_smoke.py

For architectural rationale, see docs/adr/0002-event-camera-sensor.md (ADR-0002: event-camera sensor design).

Physical EVK4 Intake

Runbook for converting a physical Prophesee EVK4 capture into the SRB event-camera validation layout. This does not require public release; current HF staging remains private.

This page is for contributors running a physical EVK4 hardware capture. If you only work with synthetic event data, you can safely skip it.

Target Layout

event_camera_physical/
  events_manifest.jsonl
  episodes/
    evk4_reference.h5
    evk4_reference.flow.npz          # event_flow labels
    evk4_reference.detections.jsonl  # detection labels
  README.md

evk4_reference.h5 must use canonical EVENT_HDF5 schema v1. Label sidecars stay next to the event file and are task-specific:

event_flow: <stem>.flow.npz with flow_xy, valid_mask, and timestamps_ns.
detection: <stem>.detections.jsonl with timestamp_ns, boxes_xyxy, and class_ids.

Capture Stub

Reserve the physical dataset layout before hardware capture:

srb dataset capture-real-evk4 \
  --task excavation \
  --policy /path/to/policy.ckpt \
  --episodes 5 \
  --output /data/srb/event_camera_physical

This command is currently rejected. srb dataset capture-real-evk4 parses its arguments and then exits through the CLI preflight (cli.preflight.evk4_unsupported) before creating any directory, manifest, or protocol output, because no validated hardware capture backend and device contract exists in this build. The layout below is the contract the command will reserve once a capture backend lands; create it by hand until then.

Label Template

Also currently rejected. Every srb dataset validate-events downstream invocation except --synthetic-smoke is refused by the CLI preflight (cli.preflight.downstream_unsupported) before writing output — including the --write-label-template and --check-real-target forms below. The template and contract-check helpers exist in-tree and are unit-tested, so the shapes documented here are the ones that will run once the downstream harness and model adapter land.

After evk4_reference.h5 exists, create a valid sidecar skeleton:

srb dataset validate-events downstream \
  --synth-events /data/srb/synth/events.h5 \
  --real-events /data/srb/event_camera_physical/episodes/evk4_reference.h5 \
  --task event_flow \
  --output /tmp/downstream_stub.json \
  --write-label-template

For event_flow, the template contains one zero-flow frame with valid_mask=False everywhere. For detection, the template contains one timestamped record with no boxes. These files are valid contract skeletons, not finished labels.

Validate once labels have been filled:

srb dataset validate-events downstream \
  --synth-events /data/srb/synth/events.h5 \
  --real-events /data/srb/event_camera_physical/episodes/evk4_reference.h5 \
  --task event_flow \
  --output /tmp/downstream_stub.json \
  --check-real-target

Private HF Staging

Current private fixture repository:

AndrejOrsula/srb-event-camera-fixtures

Upload convention for permissioned collaborators:

hf upload AndrejOrsula/srb-event-camera-fixtures \
  /data/srb/event_camera_physical/episodes/evk4_reference.h5 \
  evk4/excavation/evk4_reference.h5 \
  --repo-type dataset

Keep the repo private until redistribution terms, attribution, and curation are ready. Record SHA256, byte size, event count, resolution, task, policy id, and capture date in the dataset card or adjacent README before granting broader access.

Pull And Check

Permissioned users should fetch through the SRB fixture registry when a mirror entry exists:

srb event-camera list-fixtures
srb event-camera fetch-fixture --mirror hf-andrejorsula-active-marker-gen4

For new physical task datasets, add a distinct private mirror short name only after the SHA256 and license note have been reviewed.

Event-Camera Weight Mirrors

Three upstream checkpoints flow through the srb event-camera fetch-weights CLI; SRB redistributes none in-repo:

Readiness across weights, fixtures, validation, and Phase A / Phase B gates is summarized in the Status Matrix.

Artifact	`--artifact` value	Filename	Consumer
SuperSloMo upsampler	`supersloMo` (default)	`SuperSloMo39.ckpt` (~151 MB)	v2e backend constructed with `use_interp=True` + a SloMo model path (`cfg.frame_interp` is not currently wired through `make_backend`)
E2VID reconstruction	`e2vid`	`E2VID_lightweight.pth.tar` (~40 MB)	`srb.datasets.event_validation.reconstruction.e2vid_reconstruct` (S13 #24)
V2CE learned simulator	`v2ce`	`v2ce_3d.pt`	`srb.core.sensor.event_camera.backends.v2ce.V2CEBackend` (S8 #22)

The E2VID checkpoint comes from the original rpg_e2vid release, but SRB’s runtime adapter uses the MIT e2vid package from uzh-rpg/e2calib through the srb[event-reconstruction] extra. Do not vendor GPL model code into SRB. The private E2VID and V2CE mirrors are exercised by the opt-in Phase-B smoke at tests/integration/test_event_camera_phase_b_smoke.py.

All three artifacts now have maintainer-pinned Hugging Face mirrors. The repository is private until the release is ready; collaborators need access plus either HF_TOKEN or a local hf auth login.

Mirror	Artifact	URL SHA256
`hf-andrejorsula-supersloMo`	`SuperSloMo39.ckpt`	`d4d3070431eff774a1c27038930121ffeae59ede3a7d184118c539b443ea27cb`
`hf-andrejorsula-e2vid`	`E2VID_lightweight.pth.tar`	`4cfeb2c850bf48fc9fa907e969cb8a04e3c51314da2d65bdb81145ac96574128`
`hf-andrejorsula-v2ce`	`v2ce_3d.pt`	`97920e0c3f3c1df55f12800c166bf1de11466f7546bdb7b40261c5a14cddef26`

Original upstream sources:

SuperSloMo: https://drive.google.com/file/d/1ETID_4xqLpRBrRo1aOT7Yphs3QqWR_fx
E2VID: http://rpg.ifi.uzh.ch/data/E2VID/models/E2VID_lightweight.pth.tar
V2CE: https://drive.google.com/file/d/1-aC6CTGZgAZk3snANZ46FAGNkPzu_Scw

Bundling any of them in-repo would blow past the LFS budget and conflict with upstream redistribution terms.

The CLI still defaults to a placeholder artifact per kind when no --mirror / --url is supplied, so unverified downloads remain blocked:

$ srb event-camera fetch-weights                       # supersloMo (default)
WeightPlaceholderError: WeightArtifact 'SuperSloMo39.ckpt' is an unpinned
placeholder (url='srb-placeholder://...', sha256='000...'). Mirror the
checkpoint to an HTTP-fetchable host and pass --url + --sha256 ...

$ srb event-camera fetch-weights --artifact e2vid
WeightPlaceholderError: WeightArtifact 'E2VID_lightweight.pth.tar' is an
unpinned placeholder ...

$ srb event-camera fetch-weights --artifact v2ce
WeightPlaceholderError: WeightArtifact 'v2ce_3d.pt' is an unpinned
placeholder ...

This document is the publishing contract for adding a mirror for any of the three artifacts.

How users fetch weights today

Three paths work today; the first two scale to any of the three artifacts:

# 1. Explicit URL + SHA256 (one-off, e.g. private mirror)
srb event-camera fetch-weights --artifact supersloMo \
  --url https://example.com/path/to/SuperSloMo39.ckpt \
  --sha256 <hex digest> \
  --name SuperSloMo39.ckpt

srb event-camera fetch-weights --artifact e2vid \
  --url https://example.com/path/to/E2VID_lightweight.pth.tar \
  --sha256 <hex digest>

srb event-camera fetch-weights --artifact v2ce \
  --url https://example.com/path/to/v2ce_3d.pt \
  --sha256 <hex digest>

# 2. Short-name mirror (recommended for collaborators with HF access)
srb event-camera list-mirrors          # discover what's registered
srb event-camera fetch-weights --mirror hf-andrejorsula-supersloMo
srb event-camera fetch-weights --mirror hf-andrejorsula-e2vid
srb event-camera fetch-weights --mirror hf-andrejorsula-v2ce

The short-name path is the recommended one for end users: it carries the maintainer-verified SHA256 so users don’t need to copy-paste a hex digest. A mirror’s artifact.name identifies which kind it serves; the --mirror flag wins over --artifact when both are supplied. Private Hugging Face mirrors are fetched with a bearer token from HF_TOKEN, HUGGING_FACE_HUB_TOKEN, or the standard Hugging Face CLI token cache.

Mirror requirements

To register a mirror by appending to KNOWN_MIRRORS in srb/core/sensor/event_camera/weights.py, the URL must:

Be HTTP-fetchable via urllib.request — no Google Drive consent interstitials, no auth tokens, no rate-limited redirects. Confirm with:
```
srb event-camera fetch-weights --url <url> --sha256 <sha> --force
```
Serve the exact upstream checkpoint:
- For SuperSloMo: the v3.9 checkpoint shipped with v2e (filename SuperSloMo39.ckpt).
- For E2VID: the lightweight checkpoint from the rpg_e2vid release (filename E2VID_lightweight.pth.tar).
- For V2CE: the pretrained model from ucsd-hdsi-dvs/V2CE-Toolbox (filename v2ce_3d.pt).
Mirrors of modified or re-trained checkpoints must use a different short_name and a clearly distinct artifact.name.
Carry a stable URL — mirrors that hot-link to ephemeral storage (cloud function URLs, signed-URL tokens) MUST NOT be registered.
Be SHA256-verified — the maintainer adding the entry must have downloaded the file from the mirror and confirmed the digest matches a known-good reference.
Honour the upstream license — record license/attribution in WeightMirror.license_note so downstream consumers can audit redistribution.

PR template for adding a mirror

Open a PR that:

Appends one tuple to KNOWN_MIRRORS in srb/core/sensor/event_camera/weights.py. SuperSloMo, E2VID, and V2CE mirrors coexist in the same registry — the artifact.name field identifies which kind each entry serves:

# SuperSloMo example
KNOWN_MIRRORS = (
    WeightMirror(
        short_name="huggingface-<owner>-supersloMo",  # include artifact kind
        artifact=WeightArtifact(
            name="SuperSloMo39.ckpt",
            url="https://huggingface.co/<owner>/<repo>/resolve/main/SuperSloMo39.ckpt",
            sha256="<hex digest>",
            size_bytes=<file size>,
        ),
        license_note="MIT, © 2018 ...",
        notes="Mirrored 2026-MM-DD from v2e release X.Y.Z.",
    ),
    # E2VID example
    WeightMirror(
        short_name="huggingface-<owner>-e2vid",
        artifact=WeightArtifact(
            name="E2VID_lightweight.pth.tar",
            url="https://huggingface.co/<owner>/<repo>/resolve/main/E2VID_lightweight.pth.tar",
            sha256="<hex digest>",
            size_bytes=<file size>,
        ),
        license_note="GPL-3.0, © 2019 Rebecq et al.",
        notes="Mirrored 2026-MM-DD from rpg_e2vid release X.Y.Z.",
    ),
    # V2CE example
    WeightMirror(
        short_name="huggingface-<owner>-v2ce",
        artifact=WeightArtifact(
            name="v2ce_3d.pt",
            url="https://huggingface.co/<owner>/<repo>/resolve/main/v2ce_3d.pt",
            sha256="<hex digest>",
            size_bytes=<file size>,
        ),
        license_note="MIT, © 2024 Zhongyang Zhang.",
        notes="Mirrored 2026-MM-DD from ucsd-hdsi-dvs/V2CE-Toolbox.",
    ),
)

Use the artifact kind as a short_name suffix (-supersloMo, -e2vid, or -v2ce) so srb event-camera list-mirrors is self-documenting and typo-resistant when multiple mirrors come from the same host.

Updates the tripwire test (tests/unit/test_event_camera_weight_mirrors.py ::test_known_mirrors_pin_private_huggingface_weights and ::test_known_mirror_artifact_hashes_are_pinned) to assert the new non-empty registry shape. The tripwire is intentional — its failure forces a reviewer to read this document.
Adds a CLI smoke test that resolves the new short_name and confirms WeightArtifact.url / sha256 round-trip through get_known_mirror.
Includes evidence in the PR description that an end-to-end srb event-camera fetch-weights --mirror <short_name> succeeded against the mirror (paste the CLI JSON output).
Records the upstream license in the PR body and confirms the maintainer has authority to redistribute. Known upstream licenses: SuperSloMo / v2e is MIT, the E2VID checkpoint was published with rpg_e2vid under GPL-3.0, the E2VID Python provider used by SRB is MIT via uzh-rpg/e2calib, and V2CE-Toolbox is MIT.

Removing a mirror

If a registered mirror goes offline or serves a corrupted file, open a PR that:

Removes the entry from KNOWN_MIRRORS.
Updates the tripwire test to reflect the new shape.
Documents the reason in the commit body so the same mirror is not re-added by accident later.

Why this is a registry and not a single canonical URL

There is intentionally no “the SRB mirror” for any of the three artifacts. SRB does not own the upstream checkpoints and does not want to be the single point of failure for the event-camera pipeline. The registry pattern lets community mirrors live side-by-side with private institutional mirrors without one bucket having to scale to every SRB user.

Event-Camera Fixture Mirrors

A Gen4 reference fixture is a small Prophesee Gen4-HD event-camera recording in the frozen EVENT_HDF5 schema v1 (see ADR-0002 §12). It is the held-out evaluation target consumed by:

Readiness across fixtures, weights, validation, and Phase A / Phase B gates is summarized in the Status Matrix.

Consumer	Module	Status
Downstream-fidelity probe (S14 / #30)	`srb.datasets.event_validation.downstream.run_downstream_evaluation`	Raises `NotImplementedError` until a labelled/task-specific fixture + harness land
S15 Phase B preset-retune calibration	`srb.core.sensor.event_camera.preset`	Soft-gated (docstring); waiting on this fixture + a real EVK4 capture
S19 Phase B release-smoke fixture path	`tests/integration/test_event_camera_phase_b_smoke.py`	Opt-in; validates the private active-marker fixture and E2VID checkpoint when enabled

SRB ships no captures in git. Gen4 recordings are institutionally owned and capture-licensed; the lab/team that ran the EVK4 owns the data and decides whether to publish it. Private staging mirrors can be registered while public redistribution terms are being confirmed. The CLI default remains a placeholder fixture that the fetcher refuses to download unless a mirror or explicit URL is supplied:

$ srb event-camera fetch-fixture
FixturePlaceholderError: FixtureArtifact 'gen4_reference.h5' is an
unpinned placeholder (url='srb-placeholder://...', sha256='000...').
Mirror the capture to an HTTP-fetchable host and pass --url + --sha256
on `srb event-camera fetch-fixture`, or construct
FixtureArtifact(... url=..., sha256=...) directly. ...

This document is the publishing contract for adding or changing a Gen4 fixture mirror in KNOWN_FIXTURES.

How users fetch fixtures today

Three paths work today:

# 1. Explicit URL + SHA256 (one-off, e.g. private institutional mirror)
srb event-camera fetch-fixture \
  --url https://your-host.example/gen4_reference.h5 \
  --sha256 <hex digest> \
  --name gen4_reference.h5

# 2. Short-name mirror (recommended for collaborators with HF access)
srb event-camera list-fixtures          # discover what's registered
srb event-camera fetch-fixture --mirror hf-andrejorsula-active-marker-gen4

The short-name path is the recommended one for end users: it carries the maintainer-verified SHA256 and the pinned schema version, so users don’t need to copy-paste either.

Candidate source: Prophesee Active Marker

Prophesee’s active_marker KDrive share is a useful source capture for fixture preparation and now has a private SRB staging mirror:

Share page: https://kdrive.infomaniak.com/app/share/975517/d39fd010-8d56-4ce1-980d-224d5654b478
Downloaded ZIP SHA256: 3a2aced3efee1b37b89e53bcb8b2882985ff6da9b556e7cfa1494e42e509098f
Source payload: MV_SDK_5_x/active_marker_dataset/active_marker.raw
RAW header: EVT3, generation 4.2, geometry 1280x720
Private mirror: hf-andrejorsula-active-marker-gen4
HF dataset repo: AndrejOrsula/srb-event-camera-fixtures
HF path: prophesee/active_marker/active_marker.h5

It is registered only as a private staging mirror. Do not make the HF repo public until Prophesee redistribution terms are confirmed. Conversion evidence from srb dataset import-raw-evt3:

srb dataset import-raw-evt3 active_marker.raw \
  --output active_marker.h5 \
  --task active_marker \
  --policy-id prophesee_source

Verified converted HDF5:

Event count: 22_316_758
Resolution: 1280x720
EVT3 24-bit time wraps: 1
Invalid coordinates skipped: 0
HDF5 size: 61_565_619 bytes
HDF5 SHA256: 1e54f9853982ec0920a10aff3a5340322f435edd5ac29712bfd2db2eda47d6c0
Local size: 58.7M with gzip fallback compression

Verified private HF upload:

srb event-camera fetch-fixture \
  --mirror hf-andrejorsula-active-marker-gen4 \
  --force

This fixture is useful for schema, importer, fetcher, and statistical smoke work, and for the opt-in Phase-B E2VID smoke. It does not by itself lift the downstream-fidelity NotImplementedError; that harness still needs a labelled/task-specific real Gen4 target and model adapter.

Fixture mirror requirements

To register a fixture by appending to KNOWN_FIXTURES in srb/core/sensor/event_camera/fixtures.py, the URL must:

Be HTTP-fetchable via urllib.request — no Google Drive consent interstitials, no auth tokens, no signed-URL tokens that expire. Confirm with:
```
srb event-camera fetch-fixture --url <url> --sha256 <sha> --force
```
Serve an EVENT_HDF5 file with schema v1:
- Datasets /events/t (int64 ns), /events/x (uint16), /events/y (uint16), /events/p (int8: −1/+1).
- Group /metadata with schema_version=1, resolution, and the reproducibility attrs from EventEpisodeMetadata.
- Optionally Blosc:zstd-3 or gzip-6 compressed (the SRB reader handles both).
Mirrors of modified or resampled captures must use a different short_name and a clearly distinct artifact.name, and bump the schema_version only via an ADR amendment.
Pin the sensor model in FixtureArtifact.sensor_model — the downstream-fidelity probe consults this to pick the right resolution prior (currently "prophesee_gen4_hd"; the field accepts other strings if a future fixture targets DAVIS or EVK3).
Carry a stable URL — mirrors that hot-link to ephemeral storage (cloud-function URLs, signed-URL tokens, presigned S3) MUST NOT be registered. Zenodo, Hugging Face datasets, and institutional research data repositories with DOIs are the recommended hosts.
Be SHA256-verified — the maintainer adding the entry must have downloaded the file from the mirror and confirmed the digest matches the known-good reference.
Honour the capture license — record license/attribution in FixtureMirror.license_note (typically CC-BY-4.0 for academic captures; never a license stricter than the upstream redistribution terms permit). Note the distinction from the weight registry: captures are data not code, so the conventional license is CC-BY-4.0 rather than MIT.

PR template for adding a fixture mirror

Open a PR that:

Appends one tuple to KNOWN_FIXTURES in srb/core/sensor/event_camera/fixtures.py:

KNOWN_FIXTURES = (
    FixtureMirror(
        short_name="zenodo-<owner>-gen4",  # include sensor kind suffix
        artifact=FixtureArtifact(
            name="gen4_reference.h5",
            url="https://zenodo.org/records/<id>/files/gen4_reference.h5",
            sha256="<hex digest>",
            size_bytes=<file size>,
            schema_version=1,
            sensor_model="prophesee_gen4_hd",
            num_episodes=<count>,
        ),
        license_note="CC-BY-4.0, © 2026 <owner/lab>",
        notes="Captured 2026-MM-DD on EVK4-HD, <scene summary>.",
    ),
)

Use the sensor kind as a short_name suffix (-gen4, -davis346, -evk3) so srb event-camera list-fixtures is self-documenting and typo-resistant when multiple sensor families coexist.

Updates the tripwire test (tests/unit/test_event_camera_fixture_mirrors.py ::test_known_fixtures_registry_is_pinned) to assert the registry shape. The tripwire is intentional — its failure forces a reviewer to read this document.
Adds a CLI smoke test that resolves the new short_name and confirms FixtureArtifact.url / sha256 / schema_version round-trip through get_known_fixture.
Includes evidence in the PR description that an end-to-end srb event-camera fetch-fixture --mirror <short_name> succeeded against the mirror (paste the CLI JSON output, including the schema_version and sensor_model fields).
Records the capture license in the PR body and confirms the maintainer has authority to redistribute (institutional sign-off where applicable; subject-consent for any human-visible content).
Lifts the NotImplementedError in srb/datasets/event_validation/downstream.py::run_downstream_evaluation ONLY if this PR also wires the real harness (train + eval). A mirror-only PR keeps the NotImplementedError in place — both PRs can be authored in parallel, but the lifting commit MUST NOT land without a real verified mirror present in KNOWN_FIXTURES.

Removing a fixture mirror

If a registered fixture goes offline, serves a corrupted file, or has its license retracted, open a PR that:

Removes the entry from KNOWN_FIXTURES.
Updates the tripwire test to reflect the new shape.
Documents the reason in the commit body so the same mirror is not re-added by accident later.

Why this is a registry and not a single canonical URL

There is intentionally no “the SRB Gen4 fixture.” SRB does not own capture data and cannot be the single point of failure for the downstream-fidelity probe. The registry pattern lets institutional mirrors (university research-data repos, Zenodo deposits with DOIs) live side-by-side with community mirrors without one bucket having to scale to every SRB user — and lets multiple labs publish their own Gen4 captures without a coordination handshake.

Parallel pattern to the SuperSloMo / E2VID weight registry — see event_camera_weight_mirrors.md for the precedent and the same rationale applied to ML checkpoints.

Command Line Interface (CLI)

The Space Robotics Bench provides a comprehensive command-line interface that allows you to interact with most aspects of the framework through a unified interface. The CLI is structured as a set of subcommands, each with its own set of options.

srb <subcommand> [options]

If you need help with a specific subcommand, you can use the -h/--help flag:

srb --help
srb <subcommand> -h

You can also TAB your way through the available subcommands and options via autocompletion:

srb <TAB> <TAB>

Subcommands

The following subcommands are available in the CLI:

Subcommand	Description
`agent`	Agent subcommands (listed below)
`real_agent`	Sim-to-real bridge code generation and runtime
`dataset`	Inspect, validate, convert, replay, and publish datasets
`model`	Package, validate, and publish trained policy checkpoints
`list`/`ls`	List registered assets, action groups and environments
`asset`	Manage planetary surface assets, rock sets, sky-domes, and lighting presets
`event-camera`	Manage event-camera weights and backend metadata
`ephemeris`	Query real Sun/planet geometry, fetch SPICE kernels, import illumination products
`cache`	Regenerate generated SRB offline caches
`gui`	Launch the Graphical User Interface
`isaac`/`isaacsim`	Isaac Sim utilities (`sim`, `python`)
`docs`	Build docs and render asset/task thumbnails
`test`	Run Python and/or Rust test suites
`clean`	Remove cached files (SRB / Isaac Sim / SimForge)
`completions`	Manage shell completions (cache + install scripts)

Agent Subcommands

The agent subcommand is further separated into agent-specific subcommands:

srb agent <action> [options]

Subcommand	Purpose
`zero`	Run a simulation agent with zero-valued actions.
`rand`	Run a simulation agent with random actions.
`teleop`	Manually teleoperate the agent, directly or through a trained policy.
`ros`	Drive the simulation agent from ROS 2 / Space ROS action topics.
`train`	Train a reinforcement-learning policy in simulation.
`eval`	Evaluate a trained policy in simulation.
`collect`	Record demonstration datasets from human, policy, random, or zero controllers.
`learn`	Train an imitation-learning policy offline from a demonstration dataset.
`tune`	Optimize hyperparameters with Optuna (spawns training subprocesses per trial).

Shared Agent Options

The following options are shared across the simulation-based agent subcommands (zero, rand, teleop, ros, train, eval, collect). The offline learn and tune subcommands do not launch Isaac Sim directly (tune spawns srb agent train subprocesses that do) and accept their own option groups.

Argument	Description	Default
`-e`/`--env`/`--task` ENV	ID of the environment	REQUIRED
`--cfg` PATH	Hydra config YAML (`DEFAULT`/`IGNORE`/`NONE`/`NULL` recognised)	`DEFAULT`
`--headless`	Run simulation without display	`False`
`--hide_ui`	Disable Isaac Sim UI and force fullscreen	`False`
`--livestream` {0,1,2}	Force livestreaming (0: off, 1: native, 2: WebRTC). `-1` defers to `LIVESTREAM` env var	`-1`
`--rendering_mode` {performance,balanced,quality,xr}	Choose rendering preset	unset
`--xr`	Enable XR mode for VR/AR applications	`False`
`--enable_cameras`	Launch the rendering app for camera sensors (without video recording). Auto-enabled when the CLI activates a camera-backed sensor category (`env.active_sensors=[…,visual]` or `event_camera`), for camera-shipping demo envs, and by `--video`/`--record-events`/`--snapshot` — pass it explicitly only when none of those apply	`False`
`--kit_args` STRING	Extra args forwarded to the Omniverse Kit	`""`
`--interface` [IFACE …]	Interfaces to enable (`gui`, `ros`)	`[]`
`--logdir`/`--logs` PATH	Path to root logging directory	`SRB_LOGS_DIR`
`--video`	Enable video recording	`False`
`--record-events`/`--record_events`	Record event-camera rollouts with provenance. Scene materialization, observation delivery, and the recorder are wired, but no concrete event backend (v2e, Metavision, V2CE) ships in the default container image, so provider-backed rollout fidelity is unproven; use `srb dataset record-events` for the live synthetic path.	`False`
`--snapshot` PATH	Capture a single rendered PNG after the initial reset, then exit (implies `--enable_cameras`)	unset
`--snapshot-warmup` N	Render passes before the snapshot to let path-traced/anti-aliased accumulation converge	`16`
`--snapshot-settle` N	Physics steps after the reset and before the snapshot, letting robots settle (`0` disables)	`0`
`--perf`	Run a performance test instead of the regular workflow	`False`
`--perf_output` PATH	Output path of the performance report (or `STDOUT`)	`STDOUT`
`--perf_duration` SEC	Maximum duration of the performance test in seconds (`0` = unlimited)	`150.0`
`--repl` [BACKEND]	Open an interactive Python REPL alongside the running simulation (`ptpython`/`bpython`)	unset

Note: Trailing CLI tokens that do not match the registered options are forwarded to Hydra (env.foo=bar, agent.lr=1e-3, …). Tokens after a literal -- are forwarded to the underlying tool (e.g. pytest, cargo).

Event Camera Subcommands

srb event-camera manages event-camera weights and backend metadata:

Sub-subcommand	Description
`srb event-camera fetch-weights`	Download event-camera weights (SuperSloMo, E2VID, or V2CE) with SHA256 verification
`srb event-camera list-mirrors`	List registered event-camera weight mirrors (SuperSloMo + E2VID + V2CE)
`srb event-camera fetch-fixture`	Download an event-camera reference fixture (Gen4 HDF5) with SHA256 verification
`srb event-camera list-fixtures`	List registered event-camera reference-fixture mirrors (Gen4 HDF5)
`srb event-camera backends`	List registered event-generator backends (use `--detailed` for metadata)

Cache Subcommands

srb cache manages generated SRB offline caches used by completions and the GUI:

Sub-subcommand	Description
`srb cache update`	Regenerate task, asset, config, enum, and GUI caches
`srb cache claim-clean-root [--target srb\|isaacsim\|all]`	Add path- and inode-bound markers required by `srb clean`; only fixed cache paths may adopt existing content
`srb cache status [--json] [--strict]`	Inspect generated cache files without launching Isaac Sim
`srb cache doctor`	Validate generated cache payload shapes without launching Isaac Sim

Completions Subcommands

srb completions manages shell-completion artefacts:

Sub-subcommand	Description
`srb completions generate`	Regenerate completion cache via `srb cache update`
`srb completions install`	Install Fish/Bash/Zsh completion scripts

Start a new shell after installation (or run exec "$SHELL" -l) so Bash and Zsh load their new registrations. Hydra override candidates require jq and a successful srb completions generate.

`srb list` (alias `srb ls`) — List Assets and Environments

The srb list command — also reachable via the srb ls alias — lists all registered assets, action groups, hardware interfaces, and environments in the Space Robotics Bench. Every invocation launches headless Isaac Sim to enumerate the registries (and refreshes the offline cache used by shell completions and the GUI), so expect the usual Isaac Sim startup time.

Usage

srb list [options]
srb ls   [options]   # alias

Options

Argument	Description	Default
*categories	Categories to list. Choices: `all`, `action`, `asset`, `env`, `scenery`, `object`, `robot`. Selecting `asset` is equivalent to `scenery object robot`.	`[all]`

Note: Templates whose name starts with _ (e.g. _aerial, _manipulation) are listed alongside concrete tasks. They are intended to be customised via Hydra overrides such as env.robot=<name>.

Output

The command produces a tabular output with the requested registered entities. The action category (included in the default all) additionally prints a Hardware Interfaces table with the registered sim-to-real interface classes, and the Assets table includes an Action Modes column listing each robot’s selectable action modes (the default mode is marked with a leading *).

Hint: In VSCode, the table is interactive and provides clickable links to the source code.

Example Output (Truncated)

                                                          Assets of the Space Robotics Bench
┏━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ # ┃ Name                        ┃ Type    ┃ Subtype            ┃ Parent Class          ┃ Asset Config    ┃ Path                             ┃
┡━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 1 │ mars_surface                │ scenery │ terrain            │ Terrain               │ AssetBaseCfg    │ terrain.py                       │
├───┼─────────────────────────────┼─────────┼────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────────┤
│ 2 │ cargo_bay                   │ object  │ payload            │ Payload               │ AssetBaseCfg    │ payload/cargo_bay.py             │
├───┼─────────────────────────────┼─────────┼────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────────┤
│ 3 │ scoop                       │ object  │ tool               │ Tool                  │ RigidObjectCfg  │ tool/scoop.py                    │
│ 4 │ shadow_hand                 │ object  │ tool               │ ActiveTool            │ ArticulationCfg │ tool/shadow_hand.py              │
├───┼─────────────────────────────┼─────────┼────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────────┤
│ 5 │ sample_tube                 │ object  │ common             │ Object                │ RigidObjectCfg  │ sample.py                        │
├───┼─────────────────────────────┼─────────┼────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────────┤
│ 6 │ franka                      │ robot   │ manipulator        │ SerialManipulator     │ ArticulationCfg │ manipulation/franka.py           │
│ 7 │ random_ur_manipulator       │ robot   │ manipulator        │ SerialManipulator     │ ArticulationCfg │ manipulation/universal_robots.py │
├───┼─────────────────────────────┼─────────┼────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────────┤
│ 8 │ random_unitree_quadruped    │ robot   │ mobile_robot       │ LeggedRobot           │ ArticulationCfg │ mobile/unitree.py                │
├───┼─────────────────────────────┼─────────┼────────────────────┼───────────────────────┼─────────────────┼──────────────────────────────────┤
│ 9 │ unitree_g1                  │ robot   │ mobile_manipulator │ Humanoid              │ ArticulationCfg │ mobile_manipulation/unitree.py   │
│ . │ ...                         │ ...     │ ...                │ ...                   │ ...             │ ...                              │
└───┴─────────────────────────────┴─────────┴────────────────────┴───────────────────────┴─────────────────┴──────────────────────────────────┘

             Action Groups of the Space Robotics Bench
┏━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ # ┃ Name                          ┃ Path                       ┃
┡━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 1 │ body_acceleration             │ common/body.py             │
│ 2 │ body_acceleration_relative    │ common/body.py             │
│ 3 │ joint_position                │ common/joint.py            │
│ 4 │ joint_position_relative       │ common/joint.py            │
│ 5 │ joint_position_binary         │ common/joint.py            │
│ 6 │ joint_velocity                │ common/joint.py            │
│ 7 │ joint_velocity_binary         │ common/joint.py            │
│ 8 │ joint_effort                  │ common/joint.py            │
│ 9 │ inverse_kinematics            │ manipulation/task_space.py │
│ . │ ...                           │ ...                        │
└───┴───────────────────────────────┴────────────────────────────┘

                                                        Environments of the Space Robotics Bench
┏━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ # ┃ ID                               ┃ Entrypoint                   ┃ Config                             ┃ Path                                      ┃
┡━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 1 │ _manipulation <template>         │ Task(ManipulationEnv)        │ TaskCfg(ManipulationEnvCfg)        │ manipulation/_manipulation                │
│ 2 │ sample_collection                │ Task(ManipulationEnv)        │ TaskCfg(ManipulationEnvCfg)        │ manipulation/sample_collection            │
│ 3 │ _aerial <template>               │ Task(AerialEnv)              │ TaskCfg(AerialEnvCfg)              │ mobile/_aerial                            │
│ 4 │ _ground <template>               │ Task(GroundEnv)              │ TaskCfg(GroundEnvCfg)              │ mobile/_ground                            │
│ 5 │ _orbital <template>              │ Task(OrbitalEnv)             │ TaskCfg(OrbitalEnvCfg)             │ mobile/_orbital                           │
│ 6 │ locomotion_velocity_tracking     │ LocomotionTask(Task)         │ LocomotionTaskCfg(TaskCfg)         │ mobile/velocity_tracking                  │
│ 7 │ _aerial_manipulation <template>  │ Task(AerialManipulationEnv)  │ TaskCfg(AerialManipulationEnvCfg)  │ mobile_manipulation/_aerial_manipulation  │
│ 8 │ _ground_manipulation <template>  │ Task(GroundManipulationEnv)  │ TaskCfg(GroundManipulationEnvCfg)  │ mobile_manipulation/_ground_manipulation  │
│ 9 │ _orbital_manipulation <template> │ Task(OrbitalManipulationEnv) │ TaskCfg(OrbitalManipulationEnvCfg) │ mobile_manipulation/_orbital_manipulation │
│ . │ ...                              │ ...                          │ ...                                │ ...                                       │
└───┴──────────────────────────────────┴──────────────────────────────┴────────────────────────────────────┴───────────────────────────────────────────┘

`srb agent zero` — Zero Agent

The srb agent zero command runs an environment with zero-valued actions. For action spaces with equilibrium at zero, the environment will evolve solely based on its internal dynamics. This is useful for testing the environment and observing the behavior of the agent without any control input.

Usage

srb agent zero --env ENV_ID [options]

Options | Shared Agent Options

No additional options are available for this subcommand.

Example

Run zero agent in the _ground template environment:

Create 4 parallel environment instances
Use the Cassie bipedal robot

srb agent zero -e _ground env.num_envs=4 env.robot=cassie

`srb agent rand` — Random Agent

The srb agent rand command runs an environment with randomly sampled actions. This is useful for testing the environment’s response to diverse inputs, identifying edge cases, and ensuring robustness in the environment implementation.

Usage

srb agent rand --env ENV_ID [options]

Options | Shared Agent Options

No additional options are available for this subcommand.

Example

Run random agent in the _aerial template environment:

Hide the simulation UI
Use the body_acc action mode (default for the Ingenuity rotorcraft)

srb agent rand -e _aerial --hide_ui env.robot.action_mode=body_acc

`srb agent teleop` — Teleoperate Agent

The srb agent teleop command allows you to manually control robots in the environment using various input devices. This is useful for testing the environment and observing the behavior of the agent under specific control inputs.

Usage

srb agent teleop --env ENV_ID [options]

Options | Shared Agent Options

Option	Description	Default
`--teleop_device` [DEV …]	Interface (`keyboard`, `spacemouse`, `ros`, `gamepad`, `haptic`)	env default, else `[keyboard]`
`--pos_sensitivity` VAL	Translation sensitivity	`1.0`
`--rot_sensitivity` VAL	Rotation sensitivity (rad/s)	`π` (≈3.14)
`--invert_controls`	Invert control schema	`True`
Teleoperation via Policy
`--algo ALGO`	Algorithm of the policy (RL or IL)
`--model MODEL`	Path to the policy checkpoint
`--obs` {state,visual}	Observation modality of the policy: selects the agent-config variant and the log tree searched for its latest checkpoint	`state`

Teleoperation Modes

The teleop subcommand supports 2 modes of operation based on the environment action space:

Direct Teleoperation
Teleoperation via Policy

Direct Teleoperation

Direct teleoperation is available for environments with action spaces that can be mapped into high-level control commands, such as the desired end-effector pose or acceleration of a mobile robot.

Examples

Teleoperate a robot in the _manipulation template environment:

Use the ur10 manipulator

srb agent teleop -e _manipulation env.robot=ur10

Teleoperate a robot in the _orbital template environment:

Use the spacemouse input teleoperation device

srb agent teleop -e _orbital --teleop_device spacemouse

Teleoperation via Policy

Some action spaces do not support direct teleoperation due to their complexity or dimensionality, such as joint-space control of a humanoid robot. In such cases, you can still teleoperate the agent by providing a trained policy (both RL and imitation-learning policies are supported; the algorithm can be any --algo accepted by srb agent eval). In this case, command-like observations are driven by the teleoperation device, and the policy generates the corresponding actions.

Examples

Reference: Reinforcement Learning Workflow
Reference: srb agent train — Train Agent

First, you need to train an RL agent using the srb agent train command. Let’s demonstrate this with the locomotion_velocity_tracking environment using the dreamer algorithm:

srb agent train --headless --algo dreamer -e locomotion_velocity_tracking env.num_envs=256

After you have successfully trained a policy, you can teleoperate the agent:

srb agent teleop --algo dreamer -e locomotion_velocity_tracking

By default, the policy will be loaded from the latest checkpoint. However, you can specify a specific checkpoint using the --model option.

srb agent teleop --algo dreamer -e locomotion_velocity_tracking --model space_robotics_bench/logs/locomotion_velocity_tracking/dreamer/...

Public Demo Launch

The excavation_demo task is tuned for public events: one environment, manual reset, long horizon, all PRD-stipulated HUD lines visible, and the high-quality RTX preset selected automatically. Operators can launch it with a single command:

srb agent teleop --env excavation_demo

Add --hide_ui to drop the Isaac Sim UI for a fullscreen presentation station:

srb agent teleop --env excavation_demo --hide_ui

The default teleop devices (spacemouse, haptic, keyboard) come from the task config, so --teleop_device is optional. Hydra overrides such as env.scenery=lunalab or env.demo_regolith_quality=visual remain available for venue and visual-tier choices.

`srb agent ros` — ROS 2 Agent

The srb agent ros command enables full control over the simulation environment through ROS 2 middleware. This allows you to integrate the Space Robotics Bench with external ROS 2 nodes and systems, making it ideal for developing and validating autonomous systems across diverse scenarios.

Usage

srb agent ros --env ENV_ID [options]

Options | Shared Agent Options

Option	Default	Description
`--ros_sync`	`false`	Wait for a fresh ROS action message before each simulation step. Use this for closed-loop external controllers such as RL, LQR, or MPC policies.
`--ros_action_timeout` SECONDS	Wait indefinitely	Maximum time to wait for a fresh action when `--ros_sync` is enabled. On timeout, the latest cached action is reused.
`--ros_unthrottled`	`false`	Disable the default non-sync wall-clock pacing and run the ROS agent loop as fast as possible.

By default, srb agent ros without --ros_sync uses paced sample-and-hold control: every simulation step consumes the latest action available at that instant, while the loop is paced to the environment agent step period (env.agent_rate). If the external controller publishes slower than that period, the same cached action can still be applied for multiple steps. --ros_sync changes the loop to action-driven stepping, while --ros_unthrottled restores the fastest-possible sample-and-hold loop.

Action command subscriptions are latest-only best-effort topics (KEEP_LAST, depth 1), so high-rate publishers do not build a reliable backlog of stale actions.

Examples

See the ROS 2 Workflow guide for detailed examples.

`srb agent train` — Train Agent

The srb agent train command trains a Reinforcement Learning (RL) agent against a registered SRB environment using one of the integrated RL frameworks (dreamer, tdmpc2, sb3_*, sbx_*, rsl_rl_ppo, skrl_*).

Usage

srb agent train --algo ALGO --env ENV_ID [options]

Options | Shared Agent Options

Option	Description	Default
`--algo` ALGO	RL algorithm to use (only RL algorithms are accepted)	REQUIRED
`--obs` {state,visual}	Observation modality: selects the agent-config variant and splits the log tree (`<algo>` vs `<algo>-visual`) so state/visual checkpoints never cross-load. `visual` requires the Phase-2 sensor profiles and errors helpfully until they exist.	`state`
`--continue_training`/`--continue`	Continue training from the latest checkpoint of the most recent run (framework-dependent; TD-MPC2 rejects it)	`False`
`--model` PATH	Continue training from a specific checkpoint (framework-dependent; for TD-MPC2 a weights-only import)	unset

Note: --continue_training and --model are mutually exclusive. To resume the most recent run pass --continue_training; to resume from an arbitrary checkpoint pass --model PATH (framework-dependent — see below).

Note: what “continue” means is framework-dependent. Upstream TD-MPC2 serializes model weights only, so SRB treats --model there as a weights import: the optimizers, replay buffer, schedules, and step counter all restart from an explicitly logged step 0, and --continue_training fails immediately as unsupported rather than pretending to resume a training state that was never saved.

Supported Algorithm Families

The parser accepts registered RL algorithm names such as dreamer, tdmpc2, sbx_ppo, sb3_ppo, rsl_rl_ppo, and skrl_ppo.

Examples

# Fresh run on the landing environment with 512 parallel envs
srb agent train --algo sbx_ppo --env landing env.num_envs=512 --hide_ui

# Continue the latest run on the same env/algo combination
srb agent train --algo sbx_ppo --env landing --continue_training

# Resume from an arbitrary checkpoint
srb agent train --algo sbx_ppo --env landing \
  --model space_robotics_bench/logs/landing/sbx_ppo/<run>/ckpt/<ckpt>

Model Artifacts

Training can package the completed run into a portable SRB model artifact by passing --model-artifact-output OUT. Add --model-artifact-dataset DATASET to record training-data provenance, or use srb model from-run after training for the same packaging workflow.

See the Reinforcement Learning Workflow for an end-to-end walkthrough including hyper-parameter overrides via Hydra and Weights & Biases logging.

`srb agent eval` — Evaluate Agent

The srb agent eval command runs a trained policy in evaluation mode against a registered environment. The policy can be from any RL or IL framework integrated with SRB (sb3_*, sbx_*, skrl_*, rsl_rl_ppo, dreamer, tdmpc2, robomimic_*).

Usage

srb agent eval --env ENV_ID (--algo ALGO | --model MODEL) [options]

Options | Shared Agent Options

Option	Description	Default
`--algo` ALGO	Algorithm of the policy to evaluate. Loads the latest checkpoint from the standard log directory unless `--model` is also given.	unset
`--model` PATH	Path to a specific checkpoint or SRB model artifact. The algorithm is inferred from the path when `--algo` is omitted.	unset
`--obs` {state,visual}	Observation modality: selects the agent-config variant and the log tree searched for the latest checkpoint (`<algo>` vs `<algo>-visual`).	`state`

Note: At least one of --algo or --model must be provided; otherwise the command exits with an error.

Supported Algorithm Families

The parser accepts registered policy algorithm names such as dreamer, tdmpc2, sbx_ppo, sb3_ppo, rsl_rl_ppo, skrl_ppo, robomimic_bc, and robomimic_cql.

Examples

# Latest checkpoint of an SBX PPO policy on landing
srb agent eval --env landing --algo sbx_ppo env.num_envs=16

# Explicit checkpoint, algorithm inferred from the path
srb agent eval --env landing \
  --model space_robotics_bench/logs/landing/sbx_ppo/<run>/ckpt/<ckpt>

# Portable SRB model artifact, including bundled restore config when available
srb agent eval --env excavation \
  --model /models/srb_excavation_kinova_gen3_smooth_osc_rsl_rl_ppo

When --model points to an SRB model artifact, evaluation uses the shared policy loader used by collect and real-world validation. That preserves bundled framework restore context such as Dreamer, TD-MPC2, or RSL-RL config.yaml files instead of depending on the original training log directory.

For a complete walkthrough see the Reinforcement Learning Workflow and the Imitation Learning Workflow.

`srb agent collect` — Collect Demonstrations

The srb agent collect command records episodes from the environment as a demonstration dataset. Data can come from human teleoperation, a trained policy, or built-in random/zero action sources. Collected datasets are consumed by srb agent learn for offline Imitation Learning (IL).

Usage

srb agent collect --env ENV_ID [options]

Options | Shared Agent Options

Option	Description	Default
Collection
`--collector` SOURCE	Data source: `human`, `policy`, `random`, or `zero`. If omitted, SRB uses `policy` when `--algo` or `--model` is present and `human` otherwise.	inferred
`--num_episodes` N	Number of episodes to collect	`100`
`--horizon` N	Maximum number of steps per episode	`400`
`--success_only`	Only save episodes that end in task success	`False`
Teleop (default mode)
`--teleop_device` [DEV …]	Input device(s) (`keyboard`, `spacemouse`, `ros`, `gamepad`, `haptic`)	env default, else `[keyboard]`
`--pos_sensitivity` VAL	Translation sensitivity	`1.0`
`--rot_sensitivity` VAL	Rotation sensitivity	`π` (≈3.14)
`--invert_controls`	Invert control schema	`True`
Policy rollout
`--algo` ALGO	RL algorithm of the trained policy
`--model` MODEL	Path to a raw policy checkpoint or SRB model artifact
`--obs` {state,visual}	Observation modality of the rolled-out policy: selects the agent-config variant and the log tree searched for its latest checkpoint	`state`

Collection Modes

1. Teleoperation (default)

With no --algo or --model, the command launches a teleoperation session. Every step recorded contains the human-generated action together with the observation. Ideal for bootstrapping a dataset when no policy exists yet.

srb agent collect --env peg_in_hole --num_episodes 50 --success_only

2. Policy Rollout

Provide --algo (and optionally --model) to replay a previously trained RL policy and record its trajectories. Supported algorithms: dreamer, tdmpc2, rsl_rl_*, sb3_*, sbx_*, skrl_*, robomimic_*.

srb agent collect --env peg_in_hole --algo sbx_ppo --num_episodes 200 --success_only

If --model is omitted, the latest checkpoint from the standard training logdir is loaded automatically. For policy collection, SRB reuses the latest run directory for that algorithm so framework-native checkpoint layouts such as model_*.pt (RSL-RL) and models/*.pt (TD-MPC2) resolve the same way as evaluation.

--model may point either to a raw checkpoint or to an SRB model artifact directory. Portable artifacts can carry framework restore config (config.yaml) beside the checkpoint; this is used by stateful or config-heavy adapters such as Dreamer, TD-MPC2, and RSL-RL when collection is replayed outside the original training run.

When collection is driven by an SRB model artifact, dataset metadata records both the resolved checkpoint path (model) and the source artifact directory (model_artifact).

3. Built-in Sources

Use --collector random or --collector zero to collect baseline datasets without a human operator or trained checkpoint.

srb agent collect --env peg_in_hole --collector random --num_episodes 20
srb agent collect --env excavation --collector zero --num_episodes 5 --horizon 200

These datasets keep the collection source separate from the dataset split, so real-world collection can still use split=real while preserving controller=random, controller=zero, controller=human, or controller=policy in artifact metadata.

Output

Datasets are written as HDF5 files compatible with robomimic. Human, random, and zero collection use ${SRB_LOGS_DIR}/<env_id>/collect/<timestamp>/datasets/; policy collection writes under the resolved policy run directory, for example ${SRB_LOGS_DIR}/<env_id>/<algo>/<timestamp>/datasets/. Run metadata records the collection source, episode controls, teleop devices, and a reproduction command. The dataset path is printed at the end of the run and can be passed directly to srb agent learn --dataset.

Tips

Use --success_only when training IL — a few clean successes outperform many failed attempts.
Collection requires env.num_envs=1 in every mode. Human, policy, random, and zero collection all raise robomimic collection requires num_envs == 1 before any dataset file is created, so set the Hydra override explicitly.
For an immersive viewpoint while teleoperating, combine with --xr (see Extended Reality).

`srb agent learn` — Learn from Demonstrations

The srb agent learn command trains a policy offline from a demonstration dataset produced by srb agent collect. Unlike the other agent subcommands, this one runs without Isaac Sim — it operates purely on the recorded dataset and therefore has a lighter footprint and faster startup.

Usage

srb agent learn --algo ALGO --env ENV_ID [options]

Options

Option	Description	Default
`-e`/`--env` ENV	ID of the environment (for logdir naming + specs)	REQUIRED
`--algo` ALGO	Imitation Learning algorithm	REQUIRED
`--dataset` PATH	Path to the demonstration dataset (HDF5). Required in practice — when omitted, auto-discovery only searches the new run’s own (empty) log directory and the run fails with a missing-dataset error	REQUIRED
`--num_epochs` N	Number of training epochs	algo default
`--normalize_actions`	Normalize actions in the dataset	`False`
`--logdir`/`--logs`	Path to logging directory	`SRB_LOGS_DIR`

Supported Algorithms

Offline learning is backed by robomimic. Algorithm names are prefixed robomimic_:

Algorithm	Description
`robomimic_bc`	Vanilla Behavioral Cloning (MLP policy)
`robomimic_bc_rnn`	Behavioral Cloning with RNN policy
`robomimic_bc_transformer`	Behavioral Cloning with Transformer policy
`robomimic_bcq`	Batch-Constrained Q-Learning
`robomimic_cql`	Conservative Q-Learning
`robomimic_iql`	Implicit Q-Learning
`robomimic_td3_bc`	TD3 with BC regularization
`robomimic_gl`	Goal-conditioned Learning
`robomimic_hbc`	Hierarchical Behavioral Cloning
`robomimic_iris`	IRIS (latent skill discovery)
`robomimic_diffusion_policy`	Diffusion Policy

See the Robomimic documentation for algorithm details and hyperparameters.

Examples

Train a Behavioral Cloning policy on a dataset collected for peg_in_hole:

srb agent learn --algo robomimic_bc --env peg_in_hole \
  --dataset logs/peg_in_hole/collect/<run>/datasets/demos_<timestamp>.hdf5

Train on a specific dataset path with an explicit epoch budget:

srb agent learn --algo robomimic_bc_rnn --env peg_in_hole \
  --dataset /path/to/demos.hdf5 --num_epochs 500

Output

Checkpoints and training metrics are written under ${SRB_LOGS_DIR}/<env_id>/<algo>/<timestamp>/. The resulting policy can be evaluated with:

srb agent eval --algo robomimic_bc --env peg_in_hole

or rolled out to produce additional demonstrations with:

srb agent collect --algo robomimic_bc --env peg_in_hole --num_episodes 200

Pass --model-artifact-output OUT to package the completed offline-learning run as an SRB model artifact. The --dataset path is used as training-data provenance unless overridden with the model-artifact dataset options.

`srb agent tune` — Hyperparameter Optimization

The srb agent tune command optimizes reinforcement-learning hyperparameters with Optuna. It runs without launching Isaac Sim itself — instead it drives an Optuna study that spawns one srb agent train subprocess per trial, so a crashing or diverging trial never takes down the study. Install the optional dependency with uv sync --extra tune.

See the Hyperparameter Tuning workflow for concepts, the wall-clock worksheet, and adoption guidance.

Verbs

srb agent tune check    [options]   # preflight a run config without training (Isaac-free)
srb agent tune run      [options]   # run or resume a study (single-task or portfolio)
srb agent tune status   --study S   # summarize a study
srb agent tune validate --study S   # re-run top-K trials on fresh seeds, re-rank (noise arbiter)
srb agent tune export   --study S   # write a tuned config to hyperparams/
srb agent tune repair   --study S   # fail stale RUNNING trials
srb agent tune baseline [options]   # capture normalization baselines (portfolio)

`run` Options

Option	Description	Default
`-e`/`--env`/`--task` ENV	Single task to tune (single-task study)	—
`--tasks` T [T …]	Task set for a portfolio study (one HP set trained on each task in turn)	—
`--algo` ALGO	Algorithm label to tune (e.g. `skrl_ppo`)	REQUIRED
`--obs` {state,visual}	Observation modality (visual requires Phase 2 profiles)	`state`
`--trial-steps` N	Total environment steps per trial, per task segment (canonical budget)	REQUIRED
`--n-trials` N	Number of finished trials to reach (resumable)	`20`
`--n-workers` N	Concurrent trial slots in this driver	`1`
`--gpus` `0,1`	GPU ids to round-robin across workers	env / GPU 0
`--trial-num-envs` N	Parallel envs per trial (pinned; drives budget conversion)	`1024`
`--seed-base` N	Base seed; child seed = `seed-base + trial*seeds-per-trial + replicate`	`42`
`--seeds-per-trial` K	Replicates per trial averaged into one lower-noise objective; pruning runs on replicate 0, so surviving trials cost ~K× (see below)	`1`
`--pin` `agent.X=Y` …	Fix config values (never sampled)	—
`--objective` {ep_return_mean,success_rate}	Objective field (tail-mean)	`ep_return_mean`
`--normalize` {none,baseline,rank}	Portfolio per-task score normalization (see below)	`none`
`--aggregate` {median,mean,min}	Portfolio cross-task aggregation	`median`
`--max-wall-hours` H	Stop asking new trials past this wall-clock budget	—
`--trial-timeout` S	Kill a trial exceeding this many seconds	—
`--fps` F	Measured env-steps/s for the wall-clock projection	—
`--n-startup` N	TPE random-startup trials before model-based sampling	`10`; `15` for portfolio studies
`--boot-stagger` S	Seconds to serialize concurrent Isaac first-boots; skipped automatically once the first child has proven its boot (metrics flowing)	`30` if `--n-workers` > 1, else `0`
`--keep-artifacts`	Keep PRUNED trials’ child logdirs (checkpoints, TB events). By default they are deleted after the prune — `trial_cfg.yaml` and `metrics.jsonl` always survive	`False`
`--retry-failed` N	Re-enqueue up to N FAILed param sets (skipping `reason=timeout`) at startup; raise `--n-trials` to give them budget to run	`0`
`--version` N	Study version; bump for an incompatible re-run	`1`
`--logdir`/`--logs`	Root directory for study artifacts	`logs/tune`
`--storage` URL	Reserved for a future RDB backend — currently errors if passed (the journal file is the only Phase-0 backend)	—

The objective field must actually be emitted by the task: a --objective success_rate study on a task that never reports success FAILs each trial early with reason success_rate_unavailable instead of training the full budget.

Averaging seeds during a study (`--seeds-per-trial`)

A single seed per trial gives the TPE sampler a noisy objective. --seeds-per-trial K trains K children per trial — each on a distinct seed (seed-base + trial*K + replicate) — and tells the sampler their mean, so the search sees a lower-variance signal. To keep the cost bounded, pruning runs on replicate 0 only: a config that looks bad on its first seed is pruned after one child, while surviving configs pay ~K× the compute. A replicate that crashes is dropped from the mean rather than failing the trial (n_seeds/n_seeds_ok are recorded on the trial). Replicate 0 keeps the canonical trials/NNN/ directory; the confirmation replicates write to trials/NNN/rep-K/ and their checkpoints are discarded. K=1 (the default) is the original single-seed behavior.

This is the during-optimization complement to validate, the post-hoc noise arbiter: use --seeds-per-trial to steer the search with a cleaner objective (at K× the budget), or run a cheap single-seed study and let validate re-rank the top handful on fresh seeds afterward.

Resuming an existing study re-validates the frozen manifest: a changed search space, --trial-steps, --objective, --obs, or task set is refused with a suggestion to bump --version (mixed budgets/objectives would corrupt the study history). Only seed-base and Optuna-version drift are tolerated.

`check` Options

check runs every cheap check run performs before its first trial — resolve the algorithm adapter and its search space, load the base agent config, check --pin format and reserved-key overlap — and then dry-assembles a single trial doc in memory (exercising the search-space sampler, config merge, pin↔sampled-key collisions, and budget conversion). It launches no study and no srb agent train child, so a misconfigured multi-hour run fails in milliseconds instead of after the first Isaac boot. It prints one [OK]/[WARN]/[FAIL] line per check and exits non-zero if any check is a hard error.

It accepts the config-shaping subset of run’s options: -e/--env/--task or --tasks, --algo (REQUIRED), --obs, --trial-steps (REQUIRED), --pin, --trial-num-envs, --seed-base, --normalize, --aggregate, --objective, --fps, --n-trials, --n-workers, --version, --logdir. For a --normalize baseline portfolio it also verifies a floor/ref baseline is present and fresh for every task. --fps adds the same wall-clock projection run prints.

srb agent tune check --env sample_collection --algo skrl_ppo --trial-steps 10_000_000 --fps 50000
srb agent tune check --tasks sample_collection peg_in_hole --algo skrl_ppo \
  --trial-steps 10_000_000 --normalize baseline   # also checks baselines.yaml

`validate` Options (noise arbiter)

TPE optimizes on a single seed per trial, so best_trial can be a seed-lucky outlier — a config that scored high by chance at the trial budget, not because it is genuinely better. validate arbitrates that noise: it re-runs the top-K completed trials with several fresh seeds each (disjoint from the training seeds) and re-ranks them by the multi-seed mean objective. It is advisory — it never writes into the study; results go to <study>/validation.json and a printed table, and you export the robust winner yourself.

Each candidate is re-run faithfully from its recorded trials/NNN/trial_cfg.yaml (the exact trained config, pins included); only the seed changes, and the budget is recomputed for --validate-steps. A portfolio candidate re-runs every task segment per seed and re-aggregates with the study’s own normalization. A candidate must COMPLETE at least --min-ok seeds (default: a strict majority) to be eligible as the winner — a config that crashes on most fresh seeds is not robust, however high its surviving-seed mean, and is flagged (below min-ok) in the table.

Option	Description	Default
`--study` NAME	Study to validate	REQUIRED
`--top-k` K	Re-run this many top trials (ranked by training value)	`3`
`--seeds` S	Fresh seeds per candidate; the objective is averaged across them	`3`
`--seed-base` B	Validation seed base (kept disjoint from training); seed = `B + i`	`10000`
`--min-ok` N	Min COMPLETEd seeds for a candidate to be eligible as winner	majority of `--seeds`
`--validate-steps` N	Env-steps per re-run; raise above the trial budget for higher fidelity	study budget
`--gpus` `0,1`	GPU ids to round-robin across re-runs	env / GPU 0
`--n-workers` N	Concurrent re-run slots	`1`
`--trial-timeout` S	Kill a re-run exceeding this many seconds	—
`--keep-artifacts`	Keep re-run child logdirs (checkpoints, TB events)	`False`
`--logdir`/`--logs`	Root directory for study artifacts	`logs/tune`

# Re-run the 5 best trials on 4 fresh seeds each; flag if the ranking flips.
srb agent tune validate --study srb-skrl_ppo-state-sample_collection-v1 \
  --top-k 5 --seeds 4 --gpus 0,1

The printout ranks candidates by validated mean and names the robust winner; when that differs from the study’s best_trial, it says so and prints the export command for the robust choice.

`status` / `export` / `repair` Options

Option	Applies to	Description	Default
`--study` NAME	all	Study name (e.g. `srb-skrl_ppo-state-pick-v1`)	REQUIRED
`--trial` N	`best`	export	Trial to export
`--to` {task,global}	export	Target slot: task-specific (L1) or global (L0). Defaults to `task` for single-task studies, `global` for portfolio studies; `--to task` on a portfolio study is ambiguous and errors	study-dependent
`--dry-run`	export	Print the diff, write nothing	`False`
`--force`	export	Overwrite an existing file	`False`
`--stale-hours` H	repair	Fail RUNNING trials older than this	`24`

`baseline` Options

Portfolio normalization (--normalize baseline) needs a per-task floor/ref pair. srb agent tune baseline captures them into hyperparams/tune/baselines.yaml.

Option	Description	Default
`-e`/`--env`/`--task` / `--tasks`	Task(s) to baseline	—
`--algo` ALGO	Algorithm label the baselines are measured for	REQUIRED
`--obs` {state,visual}	Observation modality	`state`
`--budget`/`--ref-steps` N	Reference-run env-steps (== the trial budget you will tune at)	REQUIRED
`--floor-steps` N	Random-rollout env-steps for the floor	`200000`
`--trial-num-envs` N	Parallel envs (pinned; drives budget conversion)	`1024`
`--seed-base` N	Base seed; task k uses `seed-base + k`	`42`
`--gpus` `0,1`	GPU ids to round-robin	env / GPU 0
`--objective` {ep_return_mean,success_rate}	Field floor/ref are measured on	`ep_return_mean`
`--trial-timeout` S	Kill a floor/ref run exceeding this many seconds	—
`--force`	Overwrite existing baseline entries	`False`
`--dry-run`	Compute but do not write `baselines.yaml`	`False`

Each entry records floor (random-policy mean return), ref (reference-run tail-mean), the source config path, and a git blob SHA of that config file. tune run and tune check warn when the recorded SHA no longer matches the current file — a signal to re-run tune baseline before trusting normalization.

Registered search spaces

State-observation search spaces are registered for online RL labels with global hyperparameter files. Search-space presence does not prove runtime support:

Framework	Labels with a search space
skrl	`skrl_ppo`; `skrl_ppo_rnn` and `skrl_sac` remain reserved and fail closed before runtime
stable-baselines3	`sb3_ppo`, `sb3_sac`, `sb3_tqc`, `sb3_td3`, `sb3_crossq`
sbx (JAX SB3)	`sbx_ppo`, `sbx_sac`, `sbx_tqc`, `sbx_td3`, `sbx_crossq`
rsl_rl	`rsl_rl_ppo`
DreamerV3	`dreamer`
TD-MPC2	`tdmpc2`

If you pass an --algo without a registered space, the driver lists the available ones and exits. A registered but unavailable label still fails its capability preflight. Visual-observation spaces are phased in later.

DreamerV3 / TD-MPC2 caveats

These two frameworks consume their config differently from the plain-YAML frameworks, so two limits apply:

Global (L0) tuning only. Their integrations apply the global config and silently ignore the per-task blocks inside dreamerv3.yaml / tdmpc2.yaml, so a task-specific (L1) base is not honored for them yet. --to task still writes a task-scoped export (at hyperparams/task/<task>/dreamerv3.yaml / tdmpc2.yaml — the slot the config catalog resolves), but the framework will not read it until the integration learns to extract task blocks. The plain-YAML frameworks support L1 fully.
Budget calibration. --trial-steps is passed through as the framework’s native total-env-step budget (dreamer’s nested run.steps, tdmpc2’s steps); those keys are driver-owned — export resets them to the production base, and a --pin on them errors, exactly as for the other frameworks. With num_envs > 1 the framework’s internal step counter and the reporter’s env-step count can differ by a constant factor — calibrate --trial-steps against a short reference run before trusting absolute wall-clock projections.

The DreamerV3 space tunes run.train_ratio and a model_size capacity preset (the same rssm/depth/units ladder documented in dreamerv3.yaml). The TD-MPC2 space tunes the optimizer, MPPI-planner, and loss-coefficient knobs; the architecture dims are left to model_size (TD-MPC2 derives them at load).

Portfolio (Multi-Task) Studies

Passing --tasks A B C … tunes one hyperparameter set against a whole task set: each trial trains that set sequentially on every task (fixed order), one srb agent train child per task segment under trials/NNN/seg-<task>/. The per-task scores are normalized and aggregated into a single objective; a diverging segment prunes the whole trial, and the running aggregate is reported at each segment so MedianPruner can cut weak candidates between tasks.

Because tasks have different reward scales, aggregating raw returns (--normalize none, the default) is only meaningful when they already share a scale — the driver warns otherwise. Two normalizers make scores comparable:

--normalize baseline — (score − floor) / (ref − floor) per task, from hyperparams/tune/baselines.yaml (capture it with tune baseline first). A missing baseline fails fast; a stale one (config SHA drift) warns.
--normalize rank — per-task fractional rank across completed trials; needs no baselines but is non-stationary early, so it is off by default.

--aggregate median (default) is robust to one outlier task; min is the robustness aggregate (one broken task sinks the trial); mean is the plain average. A portfolio study exports to the global L0 slot by default.

# 1. Capture floor/ref baselines for the task set.
srb agent tune baseline --tasks sample_collection peg_in_hole landing \
  --algo skrl_ppo --budget 10_000_000

# 2. Run the portfolio study, normalized against those baselines.
srb agent tune run --tasks sample_collection peg_in_hole landing --algo skrl_ppo \
  --trial-steps 10_000_000 --n-trials 30 --normalize baseline --aggregate median

# 3. Export the winner to the global per-algorithm config.
srb agent tune export --study srb-skrl_ppo-state-multi-<hash8>-v1   # --to global by default

Examples

Single-task study on sample_collection with a smoke budget, two workers on two GPUs:

srb agent tune run --env sample_collection --algo skrl_ppo \
  --trial-steps 20_000_000 --n-trials 40 --n-workers 2 --gpus 0,1

Inspect progress, then export the best trial to the task-specific slot:

srb agent tune status --study srb-skrl_ppo-state-sample_collection-v1
srb agent tune export --study srb-skrl_ppo-state-sample_collection-v1 --to task --dry-run
srb agent tune export --study srb-skrl_ppo-state-sample_collection-v1 --to task

Pin a value out of the search (and fix a task-side override):

srb agent tune run --env peg_in_hole --algo skrl_ppo --trial-steps 10_000_000 \
  --pin agent.agent.grad_norm_clip=1.0 env.robot=franka

Output

Study artifacts live under logs/tune/<study-name>/:

logs/tune/srb-skrl_ppo-state-sample_collection-v1/
  journal.log        # Optuna JournalStorage (NFS-safe; shared by parallel workers)
  study.yaml         # frozen manifest (search-space hash, budget, objective, seed base)
  trials/000/        # per-trial: trial_cfg.yaml, metrics.jsonl, logs/

Exported configs land in hyperparams/<framework>/<algo>.yaml (global) or hyperparams/task/<task>/<framework>/<algo>.yaml (task-specific) with a provenance header and are adopted via ordinary git review.

`srb real_agent` — Sim-to-Real Bridge

The srb real_agent family inspects simulated Gymnasium environments from the Space Robotics Bench, generates their lightweight real-world counterparts (registered under srb_real/), and exposes the same agent verbs (zero, rand, teleop, ros, train, eval, collect) for hardware execution. The generated environments do not depend on Isaac Sim and can be installed on the deployment target alongside the user’s chosen hardware backends.

Sub-Subcommands

srb real_agent <action> [options]

Subcommand	Purpose
`gen`	Generate the real-world counterpart for a simulated environment.
`sim2real_gen`	Alias for `gen`.
`zero`	Run a real agent with zero-valued actions.
`rand`	Run a real agent with random actions.
`teleop`	Manually teleoperate the real agent, directly or through a trained policy.
`ros`	Drive the real agent from ROS 2 / Space ROS action topics.
`train`	Train a reinforcement-learning policy directly against hardware.
`eval`	Evaluate a trained policy on hardware.
`collect`	Record real-world demonstration datasets from human, policy, random, or zero controllers.
`validate`	Run a structured real-hardware validation protocol with telemetry and pass/fail output.
`rebuild-summary`	Rebuild validation `summary.json` and `report.md` from a session directory.
`replay-twin`	Open a Rerun replay comparing a validation session against a baseline `.rrd`.

`srb real_agent gen`

srb real_agent gen --env ENV_ID [options]

Option	Description	Default
`-e`/`--env`/`--task` ENV_ID	Name of the simulated environment to generate the real-world counterpart for. Can be `ALL` to generate for every registered (non-template) environment.	REQUIRED
`--cfg` PATH	Hydra config YAML override (`DEFAULT`/`IGNORE`/`NONE`/`NULL` recognised)	`DEFAULT`
`--hardware`/`--hw` [HW …]	Default hardware interfaces (for example, `ros_cmd_vel`, `ros_tf`, `ros_mw`, `ros_depth_heightmap`, `ros_imu`) to link with the generated environment	`[]`
`--check`	Deterministic freshness gate: render a candidate, byte-compare it with the checked-in module, print a unified diff, and exit non-zero on drift without overwriting	`False`

The command inspects the Gymnasium spec of ENV_ID, asks the RealEnvGenerator for a real-world counterpart, and writes the resulting Python module to srb/interfaces/sim_to_real/env/<env_id>.py. From that point on, the same --env ID is recognised by every other srb real_agent subcommand.

Example

srb real_agent gen --env waypoint_navigation env.robot=leo_rover \
  --hardware ros_cmd_vel ros_tf ros_mw

If the environment passed to a non-gen subcommand has not yet been registered under srb_real/, the runner triggers the generation step automatically (in a sub-process) and asks you to re-run the original command.

`--check` — the freshness gate

--check never writes. It renders a fresh candidate beside the target, formats it, byte-compares it against the checked-in module, prints a unified diff, and exits non-zero on drift. Because the comparison is against a formatted file, it requires the repository’s pinned formatter (ruff) on PATH; without one it raises a typed FormatterUnavailableError naming the pinned formatter rather than reporting drift that an unformatted candidate would manufacture.

--env ALL --check runs the gate across every cached environment and exits non-zero if any has drifted — or if the environment cache is empty, since a gate that inspected nothing is not a pass. Generated modules whose task is no longer registered are absent from the cache and therefore outside the batch’s reach.

Landscaping task IDs

Terrain landscaping has two task IDs, and each generates its own module. They must never be conflated: the general task has no fixed target manifest and must never be relabelled with the crater’s hash.

Task ID	Target manifest	Generated module
`terrain_landscaping_crater`	fixed crater manifest, hash pinned in the module	`srb/interfaces/sim_to_real/env/terrain_landscaping_crater.py`
`terrain_landscaping`	per-episode mission manifest; the module declares `REQUIRES_RUNTIME_TARGET_MANIFEST = True`	`srb/interfaces/sim_to_real/env/terrain_landscaping.py`

srb real_agent gen --env terrain_landscaping_crater
srb real_agent gen --env terrain_landscaping

srb real_agent gen --env terrain_landscaping_crater --check
srb real_agent gen --env terrain_landscaping --check

Both run unqualified — no env.particles_height override belongs in them. Generating a bridge module does not make either task deployable: construction of the landscaping RealEnv still fails closed on missing hardware capability tags. See Terrain Landscaping Training Readiness.

Shared Options for the Other Sub-Subcommands

The non-gen real_agent subcommands accept the same baseline options:

Option	Description	Default
`-e`/`--env`/`--task` ENV	Name of the real environment (resolved under `srb_real/`)	REQUIRED
`--cfg` PATH	Hydra config YAML override	`DEFAULT`
`--hardware`/`--hw` [HW …]	Hardware interfaces to instantiate at runtime	`[]`
`--logdir`/`--logs` PATH	Path to root logging directory	`SRB_LOGS_DIR`

Additional groups depend on the subcommand:

teleop and collect accept the standard teleop arguments (--teleop_device, --pos_sensitivity, --rot_sensitivity, --invert_controls). Teleop devices include keyboard, spacemouse, ros, gamepad, and haptic.
collect accepts the standard collection arguments (--collector, --num_episodes, --horizon, --success_only). --collector can be human, policy, random, or zero.
train requires --algo (RL). Supported RL choices include dreamer, tdmpc2, and sbx_ppo; training also supports --continue_training / --model.
eval requires --algo and/or --model. Evaluation choices include dreamer, tdmpc2, sbx_ppo, robomimic_bc, and robomimic_cql.
teleop and collect also accept the optional Teleop Policy group (--algo, --model) for policy-driven rollouts.

eval and policy-driven collect accept SRB model artifact directories as --model values, not only raw checkpoint files. The manifest supplies the algorithm when --algo is omitted, and bundled framework config is reused by adapters that need the original restore context. Real collection records datasets with split=real while keeping the source as controller=human, controller=policy, controller=random, or controller=zero; artifact-backed policy collection records both the resolved checkpoint path and the source model artifact path.

Real Validation

srb real_agent validate --spec SPEC_PATH [options]

Option	Description	Default
`--spec` PATH	Path to validation spec YAML (e.g. `hyperparams/validation/excavation.yaml`)	REQUIRED
`--checkpoint` PATH	Override the `checkpoint` field in the spec	`None`
`--episodes` N	Override the `n_episodes` field in the spec	`None`
`--wandb`	Enable the optional Weights & Biases summary sink	`False`
`--no-tensorboard`	Disable the TensorBoard scalar sink	`False`
`--no-rerun-web`	Disable the Rerun web viewer (local `.rrd` recording is still active)	`False`
`--post-status`	Post the validation pass/fail status as a GitHub commit status (requires `gh` CLI)	`False`
`--storage-root`	Override the session artifact root directory	`logs/real_validation`
`--dry-run`	Skip environment and policy instantiation to smoke-test the validation wiring	`False`

--dry-run never passes. It skips environment and policy instantiation, then finalizes the session with exit_code: 4 (preflight_fail), pass_overall: false, hardware_evidence: false, metrics.n_episodes: 0, and a red badge.json of {"label": "real-eval", "message": "non-hardware evidence", "color": "red"}. All three criteria are reported with status skipped. A dry-run artifact is a wiring smoke test, never a validation result. See Sim-to-Real Transfer for the full exit-code table.

Validation runs the generated real environment, creates telemetry in logs/real_validation/, and calls a policy reset hook at the start of every episode when the loaded policy exposes one. --checkpoint can point to a raw checkpoint or an SRB model artifact; artifact checkpoints reuse the same policy-loading path as eval and collect, including bundled framework config for Dreamer, TD-MPC2, and RSL-RL. When the validation spec omits algo or uses REQUIRED_AT_RUNTIME, an artifact checkpoint supplies the algorithm from its manifest; an explicit conflicting spec algorithm is rejected.

`srb real_agent rebuild-summary`

Rebuild the summary.json and report.md artifacts for an existing real validation session directory. This is useful when the validation spec has changed, or you wish to adjust metrics or post-processing without re-running the physical hardware validation.

srb real_agent rebuild-summary SESSION_DIR

SESSION_DIR: Path to the validation session directory (which must contain a spec.yaml and logged .jsonl / .rrd telemetry).

`srb real_agent replay-twin`

Open a local Rerun viewer showing a time-aligned comparison of a real hardware validation rollout side-by-side with a simulated baseline rollout (“sim ghost” vs. “real solid”).

srb real_agent replay-twin --session SESSION_DIR --baseline BASELINE_RRD_PATH

--session SESSION_DIR: Path to the validation session directory containing rollout.rrd.
--baseline BASELINE_RRD_PATH: Path to the baseline .rrd file captured from a Phase 0 simulation run.

`srb asset` — Manage Assets

The srb asset subcommand groups together asset-management actions. It acts as the manager for real terrain assets, rock sets, HDRI skydomes, and lighting presets. It allows you to download and bake planetary surface terrains, validate local caches, manage the curated-site manifest, and monitor cache statistics.

Usage

srb asset <action> [options]

Actions

Action	Purpose
`thumbnail`	Render thumbnails for registered assets. (Deprecated; use `srb doc thumbnail` instead)
`list`	List available assets for a given kind (terrain, rocks, hdri, lights).
`info`	Show detailed catalog specifications for a specific asset target.
`doctor`	Preflight check the local cache health and preview rendering for a terrain site.
`download`	Fetch a remote planetary asset from the registry and bake it into USD/USDZ format.
`bake`	Import and bake terrain from a user-supplied Digital Elevation Model (DEM) file.
`clean`	Safely remove raw downloads, baked USD cache, or all assets of a specific kind.
`validate-manifest`	Validate the syntax of the curated planetary sites manifest (optionally checking URLs/SHAs).
`cache-stats`	Print total cache size, entry count, and path utilization across all asset types.
`stack`	Preview the resolved v2 source-composition plan for a region (layers, resolution, byte estimate) without downloading anything.
`prefetch-fixtures`	Automatically download and pre-bake all manifest terrain sites at low level of detail (`lod=low`).

Kind Choices

Many asset commands accept a KIND positional argument. The supported asset kinds are:

terrain: Planetary site Digital Elevation Models (DEMs) and orthophotos.
rocks: Procedural or photogrammetric rock sets.
hdri: High Dynamic Range Image skydomes.
lights: Curated environmental lighting presets.

`srb asset list`

List available assets registered in the catalog for the specified asset kind.

srb asset list KIND [--body {moon,mars,earth}]

KIND: Must be one of: terrain, rocks, hdri, lights.
--body {moon,mars,earth}: (Optional) Restrict the listed assets to a specific celestial body.

`srb asset info`

Show detailed metadata and catalog specifications for one specific asset.

srb asset info KIND TARGET

KIND: One of: terrain, rocks, hdri, lights.
TARGET: The name of the specific site, rock set, skydome, or lighting preset to inspect.

`srb asset doctor`

Preflight check the local cache health for a terrain site. This command resolves local elevation meshes, processes coordinates, and checks the asset for rendering issues.

srb asset doctor terrain [TARGET] [options]

TARGET: (Optional) The target terrain site name to verify.
--body {moon,mars,earth}: Specify the celestial body.
--patch PATCH: Patch name to verify (default: default).
--lod {low,medium,high}: Level of Detail to verify (default: medium).
--patch-size SIZE: Patch size in meters (float).
--center X,Y: Center coordinates in meters (e.g. 100.0,200.0).
--no-ortho: Disable checking/generating the orthophoto.
--seed SEED: Random seed for procedurals (default: 0).

`srb asset download`

Fetch remote planetary asset data from registry mirrors and bake it into a ready-to-render USD/USDZ representation under the local cache.

srb asset download KIND [TARGET] [options]

KIND: One of: terrain, rocks, hdri, lights.
TARGET: The name of the specific asset target to download and bake.
--all: Fetch and bake all registered assets of this kind.
--body {moon,mars,earth}: (Terrain only) Celestial body name.
--patch PATCH: (Terrain only) Patch name (default: default).
--lod {low,medium,high}: (Terrain only) Level of Detail (default: medium).
--patch-size SIZE: (Terrain only) Patch size in meters (float).
--center X,Y: (Terrain only) Center coordinates in meters.
--no-ortho: (Terrain only) Skip orthophoto baking.
--seed SEED: (Terrain only) Random seed for procedurals (default: 0).
--force: Re-download and re-bake even if a valid entry is already cached.
--debug: Enable verbose debug logging of downloading/baking pipelines.

`srb asset bake`

Bake a custom planetary surface terrain from a user-supplied local Digital Elevation Model (DEM) raster file.

srb asset bake terrain --dem DEM_PATH --body {moon,mars,earth} --site-name NAME --attribution ATTR [options]

--dem DEM_PATH: (Required) Absolute or relative path to the user-supplied DEM file (e.g., GeoTIFF).
--body {moon,mars,earth}: (Required) Target celestial body.
--site-name NAME: (Required) Name to register this custom terrain site under.
--attribution ATTR: (Required) Data source/attribution string to embed in the metadata.
--patch-size SIZE: Patch size in meters (float).
--lod {low,medium,high}: Level of Detail for the baked mesh (default: medium).
--seed SEED: Random seed for procedural rocks/displacement (default: 0).
--debug: Enable verbose debug logging during mesh generation.

`srb asset clean`

Remove cached raw download archives, baked USD files, or all local cache entries for a specific asset kind.

srb asset clean KIND [TARGET] [options]

KIND: One of: terrain, rocks, hdri, lights. Only terrain is currently supported by the clean handler — the other kinds parse but exit with clean: unsupported kind.
TARGET: (Optional) Clean cache only for the specified target name.
--raw: Remove only the raw downloaded zip/tar archives.
--baked: Remove only the processed, baked USD/USDZ asset cache.
--all: Remove all cache entries (both raw and baked) for the specified kind/target.

`srb asset validate-manifest`

Validate the schema and integrity of the curated planetary sites manifest file.

srb asset validate-manifest [--manifest MANIFEST_PATH] [--check-urls] [--check-shas]

--manifest MANIFEST_PATH: (Optional) Path to a custom manifest YAML. If omitted, uses the default packaged manifest.
--check-urls: Send HTTP requests to verify that all remote download mirrors and links are live and reachable.
--check-shas: Verify the SHA256 checksums of any downloaded raw datasets against the manifest declarations.

`srb asset cache-stats`

Display human-readable usage statistics for the local asset cache, including directory paths, file counts, and total disk space used by each asset kind.

srb asset cache-stats

`srb asset stack`

Preview the resolved real-terrain-v2 CompositionPlan for a region: which catalog layers would be used, at what resolution, how many bytes they would cost to fetch, and which of them still need consent — before anything downloads. This command never contacts the network and never moves a byte; it is a pure, offline preview over the already-loaded source catalog.

srb asset stack --body {moon,mars,earth} --lat LAT --lon LON --size SIZE [options]

--body {moon,mars,earth}: (Required) Target celestial body.
--lat LAT: (Required) Latitude in degrees.
--lon LON: (Required) Longitude in degrees.
--size SIZE: (Required) Region size in meters (capped at 8000 m; larger requests are refused with a pointer to the deferred tiling design).
--lod {low,medium,high}: Level of Detail (default: medium).
--target-gsd GSD: Override the target ground-sample distance in meters.
--catalog PATH: Override the shipped catalog with a YAML file at PATH (mainly for testing against a fixture catalog).
--allow-large: Preview the plan as if large downloads were approved (this command never downloads).
--json: Emit the plan as JSON instead of a formatted table.

The printed “consent required” line is a descriptive preview, not an authorization: the real, per-layer enforcement gate lives in the fetch path (srb.terrain.sources.acquisition.check_layer_consent) and is re-derived independently at fetch time. Seeing “consent required: none” here — whether because --allow-large was passed or because SRB_TERRAIN_LARGE_OK=1 is set — does not skip or perform any fetch.

`srb asset prefetch-fixtures`

Prefetch all curated terrain sites from the manifest at a low Level of Detail (lod=low). This is highly recommended to warm up local caches and allow headless or offline test suites to run immediately.

srb asset prefetch-fixtures

Examples

# List all registered lunar terrain sites
srb asset list terrain --body moon

# Get detailed specifications for a specific Martian terrain site
srb asset info terrain jezero

# Download and bake Jezero Crater at high level of detail
srb asset download terrain jezero --lod high

# Bake a custom terrain from a local GeoTIFF file
srb asset bake terrain \
  --dem /path/to/my_dem.tif \
  --body moon \
  --site-name Shackleton_Ridge \
  --attribution "LOLA / Kaguya Team" \
  --lod high

# Preflight check local cache for Jezero Crater terrain
srb asset doctor terrain jezero --lod high

# Clean only the baked USD terrain cache
srb asset clean terrain --baked

# Check total disk space consumed by the SRB asset cache
srb asset cache-stats

`srb dataset` — Manage Dataset Artifacts

The srb dataset command inspects, validates, converts, previews, and publishes SRB dataset artifacts. It works with canonical SRB JSONL datasets, robomimic HDF5 datasets, LeRobot-style layouts, and real-validation telemetry converted into datasets.

Usage

srb dataset <action> [options]

Actions

Action	Purpose
`info`	Print the SRB dataset manifest summary, artifact hash, and action-schema summary as JSON.
`validate`	Validate the manifest and referenced files. Declared counts are always checked against physical lengths; use `--deep` for additional row/table structure checks and `--strict` to treat warnings as errors.
`compatibility`	Report dataset compatibility for learning, collection/eval, or publication. Add `--algo ALGO` for a focused report, or omit it for the matrix.
`export`	Convert between supported layouts: `srb_jsonl`, `robomimic_hdf5`, and `lerobot`.
`pull`	Download an SRB dataset artifact from Hugging Face Hub. Resolve by `--repo-id` or `--task`/`--embodiment`/`--action-mode` identity. Use `--dry-run` to inspect the plan without network access.
`push`	Upload or dry-run a Hugging Face dataset publication. Use `--dry-run` before network upload.
`card`	Write a Hugging Face `README.md` dataset card.
`replay`	Preview a canonical dataset as a summary or with Rerun via `--backend rerun`.
`from-validation`	Convert a real validation session into a canonical SRB dataset with `split=real`.

Event-camera dataset capture and validation commands are tracked in the event-camera status matrix because some paths are live while real-fixture and Phase B validation work remains gated. The parser exposes record-events, validate-events, capture-real-evk4, export-lerobot-v3, export-zenodo-bundle, export-raw-evt3, and import-raw-evt3, and export-rosbag; nested validate-events actions are stats, reconstruction, and downstream. For the downstream stub, --synthetic-smoke runs the no-real-data adapter contract, --write-label-template creates the task-specific sidecar skeleton, and --check-real-target validates the physical EVK4 EVENT_HDF5 plus the sidecar before the future train/eval harness consumes it.

Publication Format

Canonical srb_jsonl is the preferred SRB publication format. lerobot is also publishable when the optional data dependencies are installed. Framework-native robomimic_hdf5 remains useful for training, but srb dataset push rejects it until it is exported to a publication format.

Action Metadata

Datasets store actions as flat vectors for broad framework compatibility. When the source environment exposes grouped actions, SRB records metadata.action_schema with the ordered component names, paths, shapes, dimensions, and flat-vector offsets. srb dataset info surfaces this as top-level action_dim, action_components, and action_schema fields so the action embodiment is easy to inspect without parsing raw metadata. srb dataset export preserves this field across canonical SRB JSONL, robomimic HDF5, and LeRobot layouts, and srb dataset validate checks that the declared schema agrees with metadata.action_dim. LeRobot exports also copy it into meta/info.json as srb_action_schema, so the action map remains visible to tools that read the LeRobot metadata directly; deep validation rejects mismatches between that field and the SRB manifest. Generated Hugging Face dataset cards include a compact action-schema table when this metadata is present. srb dataset replay prints the schema with the summary backend and logs per-component action values under actions/<component> when using the rerun backend.

Validation always reconciles declared episode/sample/action counts with the physical JSONL, HDF5, or Parquet lengths. If a format reader such as h5py or the pandas Parquet backend is unavailable, count validation fails explicitly; it never reports an unchecked dataset as valid.

Examples

# Validate a canonical dataset thoroughly before publishing
srb dataset validate /data/srb_excavation --deep --strict

# Convert a robomimic dataset to canonical SRB JSONL
srb dataset export /logs/excavation/collect/datasets/demos.hdf5 \
  --output /data/srb_excavation_policy --format srb_jsonl

# Preview a dataset with Rerun and save the replay
srb dataset replay /data/srb_excavation_policy \
  --backend rerun --save /tmp/srb_excavation.rrd

# Inspect upload contents without network access
srb dataset push /data/srb_excavation_policy --dry-run

# Resolve a Hub repo from artifact identity and inspect the plan
srb dataset pull --task excavation --embodiment kinova_gen3 \
  --action-mode smooth_osc --dry-run

# Download an SRB dataset artifact by repo-id
srb dataset pull --repo-id space_robotics_bench/srb_excavation_kinova_gen3_smooth_osc \
  --output /data/srb_excavation_policy

# Turn a real validation session into a dataset
srb dataset from-validation logs/real_validation/excavation/<session> \
  --output /data/srb_excavation_real \
  --embodiment kinova_gen3 --action-mode smooth_osc

from-validation keeps real-world data in split=real. By default, sessions with a validation algorithm or checkpoint are labeled controller=policy. Supported controller labels are human, policy, random, zero, and real.

`srb model` — Manage Model Artifacts

The srb model command packages trained checkpoints as self-contained SRB model artifacts. Artifacts carry SRB identity, the packaged checkpoint, training-dataset provenance, optional metrics, and framework restore metadata when needed.

Usage

srb model <action> [options]

Actions

Action	Purpose
`create CHECKPOINT --output OUT`	Package one checkpoint into a model artifact. Requires `--algo`; task, embodiment, and action mode can be inferred from `--dataset`.
`from-run RUN --output OUT`	Package the latest checkpoint from a completed SRB run. The algorithm can be inferred from `metadata.json`.
`info PATH`	Print the model manifest as JSON.
`validate PATH`	Validate the artifact manifest, checkpoint, and bundled config paths. Use `--strict` to treat warnings as errors.
`card PATH`	Write a Hugging Face `README.md` model card.
`push PATH`	Upload or dry-run a Hugging Face model publication.

Portable Restore

srb model from-run preserves framework restore context when the run contains it. Dreamer and TD-MPC2 artifacts can carry config.yaml; RSL-RL artifacts can carry the saved agent config from metadata.json or a nearby Hydra config. srb agent eval, srb agent collect, srb real_agent eval, and srb real_agent validate can use the artifact directory anywhere a checkpoint path is accepted.

Publication requires training-dataset provenance, either inferred from --dataset or supplied as --dataset-repo-id plus an optional --dataset-revision.

Examples

# Package the latest checkpoint from a run and infer identity from the dataset
srb model from-run ${SRB_LOGS_DIR}/excavation/rsl_rl_ppo/<run> \
  --dataset /data/srb_excavation_policy \
  --output /models/srb_excavation_rsl_rl_ppo

# Validate and inspect the artifact
srb model validate /models/srb_excavation_rsl_rl_ppo --strict
srb model info /models/srb_excavation_rsl_rl_ppo

# Dry-run the Hugging Face upload, including the generated README.md
srb model push /models/srb_excavation_rsl_rl_ppo --dry-run

# Reuse the packaged artifact for policy rollout
srb agent collect --env excavation --model /models/srb_excavation_rsl_rl_ppo

`srb isaac` — Isaac Sim Utilities & Python REPL

The srb isaac (alias srb isaacsim) subcommand groups thin wrappers around the Isaac Sim launcher and Python interpreter shipped with Isaac Sim. There is no standalone srb repl subcommand — interactive Python sessions are exposed in two complementary ways:

Standalone — open the Isaac Sim Python interpreter via srb isaac python (no Isaac Sim app, no scene). Useful for scripting, exploration, and ad-hoc experimentation.
In-flight — pass --repl to any simulation-based agent subcommand (zero, rand, teleop, ros, train, eval, collect) to attach an interactive REPL alongside a running simulation in a background thread.

Usage

srb isaac sim         # Launch the Isaac Sim GUI application
srb isaac python      # Run the Isaac Sim Python interpreter

Trailing arguments after the sub-subcommand are forwarded verbatim to the underlying executable, so srb isaac python -m pip list and srb isaac sim --no-window work as expected.

srb isaac sim validates that the resolved launcher is a regular executable before starting a process. If Isaac Sim is unavailable, set ISAACSIM_PATH to the installation containing isaac-sim.sh, or put the isaacsim launcher on PATH; missing, non-file, and non-executable paths are reported as typed CLI errors.

Aliases

srb isaacsim (with no further argument) is an alias for srb isaac sim, allowing you to drop directly into the GUI with a single command.

Examples

Open the Isaac Sim Python interpreter and start scripting:

srb isaac python

import torch
from srb.tasks.manipulation.sample_collection import Task, TaskCfg

env_cfg = TaskCfg(num_envs=4)
env = Task(env_cfg)
env.reset()

for _ in range(50):
    env.step(action=torch.tensor(env.action_space.sample(), device=env.device))

Open an Isaac Sim GUI session (forwarded args go straight to Kit):

srb isaac sim --/app/window/title="My SRB Session"

In-Flight REPL with `--repl`

When debugging or iterating on a live agent, attach a Python REPL to any running simulation via --repl:

srb agent zero --env sample_collection --repl ptpython
srb agent train --algo sbx_ppo --env landing --headless --repl

The REPL runs in a background thread with the simulator namespace pre-loaded (env, sim_app, helpers like mute() / unmute() to silence the simulator stdout). Supported backends are ptpython (default) and bpython. The flag can also be enabled implicitly via the SRB_REPL environment variable.

Note: --repl is not available for the offline srb agent learn subcommand because it does not launch Isaac Sim.

`srb ephemeris` — Real Sun/Planet Geometry (Optional Feature)

The srb ephemeris command queries or downloads the real sun/planet geometry used by SRB’s optional SPICE ephemeris subsystem. It is built on NASA/JPL’s NAIF SPICE toolkit via spiceypy, which is an opt-in extra (pip install 'srb[spice]') — the parser and --help/ completion paths stay fast and Isaac-free even when the extra is not installed; only running an action imports spiceypy/numpy.

Usage

srb ephemeris <action> [options]

Actions

Action	Purpose
`download`	Fetch the pinned ~43 MB kernel bundle (see kernel bundle table) into the local cache. Idempotent — already-cached, hash-verified kernels are not re-downloaded.
`info`	Print azimuth, elevation, distance, and angular diameter for one or more bodies as seen from a given site and epoch, plus the solar-irradiance scale.
`import`	Import an external precomputed illumination product (lunarlab or PGDA) onto a terrain-patch grid as an `srb_illum/2` sidecar (see Illumination Products).

`info` Options

Option	Required	Description
`--body {moon,mars,earth}`	Yes	Observer’s home body (the surface site sits on this body).
`--lat LAT`	Yes	Site latitude in degrees, `[-90, 90]`.
`--lon LON`	Yes	Site longitude in degrees.
`--alt ALT`	No (default `0.0`)	Site altitude in meters above the mean body radius.
`--utc UTC`	Yes	ISO-8601 epoch, e.g. `2026-11-01T12:00:00`. Must fall within the `de440s` kernel coverage window (1849-12-26 to 2150-01-22).
`--bodies BODIES`	No (default `sun,earth`)	Comma-separated list of bodies to query in addition to the observer’s own body. `sun` is always included regardless of this list.

`import` Options

Option	Required	Description
`--source {lunarlab-zarr,lunarlab-npz,pgda-lpsr}`	Yes	External illumination-product format to import.
`--input PATH`	Yes	Path to the source product (a directory for `lunarlab-zarr`, a file otherwise).
`--baked-dir PATH`	No	A baked Real-Terrain-v2 patch directory; the target grid, CRS, and site lat/lon are inferred from its `meta.json`, and the output defaults to `<baked-dir>/illumination.npz`. Mutually exclusive with the explicit-target options below (exactly one mode).
`--out PATH`	Explicit-target mode only	Output `.npz` path.
`--crs CRS`	Explicit-target mode only	Target CRS (e.g. an `IAU_2015:*` code).
`--center-xy CX,CY`	Explicit-target mode only	Patch center in the target CRS, meters.
`--size-m SIZE`	Explicit-target mode only	Patch size, meters.
`--lat LAT` / `--lon LON`	Explicit-target mode only	Site latitude/longitude in degrees, used for the true-north azimuth and curvature-ramp elevation corrections.
`--gsd-m GSD`	Explicit-target mode only	Target ground sample distance, meters. `--size-m` must be an integer multiple of `--gsd-m`, or the command exits with status 2.
`--epoch-range T0,T1`	No	Keep only source epochs within this inclusive ISO-8601 UTC range. Rejected (exit status 2) together with `--source pgda-lpsr`, which carries no epoch tier.
`--epoch-stride N`	No (default `1`)	Keep every Nth epoch after range filtering. Rejected (exit status 2) together with `--source pgda-lpsr`.
`--body {moon,mars,earth}`	No (default `moon`)	Body the target terrain patch sits on.

--baked-dir and --lat/--lon are mutually exclusive — the baked patch’s meta.json is the sole georef authority in --baked-dir mode, so passing --lat/--lon alongside it exits with status 2.

Examples

Pre-fetch the kernel bundle once (subsequent info calls reuse the cache):

srb ephemeris download

Query sun and earth geometry for a Shackleton-crater-rim-like lunar south pole site:

srb ephemeris info --body moon --lat -89.66 --lon 0.0 \
  --utc 2026-11-01T12:00:00 --bodies sun,earth

Query only the sun for a historical Apollo 17 EVA timestamp:

srb ephemeris info --body moon --lat 20.19 --lon 30.77 \
  --utc 1972-12-11T19:54:57 --bodies sun

Query Mars geometry (Jezero crater) for a rover-relevant epoch:

srb ephemeris info --body mars --lat 18.44 --lon 77.45 \
  --utc 2021-02-18T20:55:00 --bodies sun

Sample info output (one line per queried body, plus the irradiance scale):

sun: az=279.043° el=1.512° dist=149685302 km diam=0.559°
earth: az=64.201° el=2.551° dist=384412 km diam=1.916°
irradiance_scale=1.0021

Import a baked lunar terrain patch’s illumination directly from a lunarlab spice_maps product (target grid/CRS/site inferred from the patch’s meta.json):

srb ephemeris import --source lunarlab-zarr \
  --input /data/lunarlab/shackleton_240m/epochs.zarr \
  --baked-dir .cache/terrain/moon/shackleton_rim/<patch-key>

Import a PGDA product-69 LPSR GeoTIFF onto an explicit target grid (no baked patch available):

srb ephemeris import --source pgda-lpsr \
  --input /data/pgda/lpsr_shackleton.tif \
  --crs "IAU_2015:30135" --center-xy 0.0,10309.969398639505 \
  --size-m 4096 --gsd-m 5.0 --lat -89.66 --lon 0.0 \
  --out /tmp/shackleton_illumination.npz

Disabling Auto-Fetch

By default, missing kernels are downloaded automatically the first time they are needed (download or info). To require an explicit pre-fetch step (e.g. air-gapped CI), disable auto-fetch and pre-populate the cache ahead of time:

export SRB_EPHEMERIS_AUTO_FETCH=0
srb ephemeris download   # run once, ahead of time, with network access

With auto-fetch disabled and kernels missing, info/programmatic queries raise KernelsMissingError naming the exact srb ephemeris download command to run.

Graphical User Interface (GUI)

SRB comes with a minimal GUI application that can serve as a more approachable demonstration for non-developers. The GUI is written in Rust using the egui framework, while the middleware between the GUI and the SRB framework is facilitated via the r2r ROS 2 bindings.

The GUI discovers its task cards and configuration options from the gui_catalog.json member of one immutable offline-cache generation. srb cache update builds all twelve cache members and their hashes under .cache/offline/generations/<generation-id>/, validates their semantic and catalog consistency, then atomically switches .cache/offline/current.json. Rust resolves that pointer and validates the manifest and every required member before reading the catalog; it never reads the legacy fixed files directly.

Recommended demo defaults are:

rendering mode: quality
teleop devices: keyboard spacemouse
interface: gui
task-specific env.stack from the generated catalog (excavation_demo uses true)

Haptic devices are not configured through dedicated GUI knobs. If a haptic teleop device is available in the SRB cache, it appears only as a generic checkbox under advanced controls.

The customization panel shows the exact command before launch and provides both run and copy actions. Argparse options are emitted first; Hydra overrides are appended as key=value tokens without a -- separator.

`srb gui` — Launch GUI

The srb gui command launches the GUI through cargo run when the native Rust toolchain is available:

srb gui

Hint: All arguments following -- are forwarded to the GUI application itself.

The command checks for cargo and rustc immediately before launching. If the toolchain is unavailable, use an executable prebuilt GUI instead:

SRB_GUI_BINARY=/path/to/gui srb gui

In the development tree, an executable target/release/gui or target/debug/gui is also accepted as the prebuilt path. Missing or invalid launch inputs produce an actionable typed CLI error before a process is started.

Example

Print the help message of the GUI application:

srb gui -- --help

Refresh GUI Catalog

Regenerate the complete offline snapshot after changing task metadata, cache generation, assets, task thumbnails, or available interfaces:

srb cache update

srb cache update always launches Isaac Sim because task and registry discovery are runtime-backed. In a lightweight environment, use srb cache status or srb cache doctor to inspect an already-generated catalog without launching Isaac Sim; neither command regenerates missing metadata. A missing, incomplete, or tampered pointer/generation is reported as incomplete and must be repaired explicitly with srb cache update. The older .cache/*.json files and .cache/generation.json, if present, are compatibility artifacts only and do not make the offline snapshot ready.

Task Thumbnails

Task thumbnails are loaded from:

docs/src/_graphics/tasks/<task>.webp

Generate or refresh a single task thumbnail with:

srb doc thumbnail --task excavation_demo --force --width 320 --height 240

Missing task thumbnails are not fatal. The catalog falls back to a robot thumbnail, scenery thumbnail, domain thumbnail, then the SRB placeholder.

Command Shape

The GUI command preview follows SRB CLI parsing rules. For example:

srb agent teleop --env excavation_demo --hide_ui --rendering_mode quality --interface gui --teleop_device keyboard spacemouse env.domain=MOON env.seed=0 env.num_envs=1 env.stack=true env.robot=kinova_gen3n7+scoop_custom3

Use raw Hydra overrides in the customization panel for task-specific options such as env.demo_regolith_quality=visual.

Compatibility Matrix

The matrix below shows which robot category each task supports. A ✓ marks a supported combination; a blank cell means it is not available. Tasks prefixed with _ (e.g. _manipulation) are reusable base templates rather than directly runnable scenarios.

Scope. This matrix is about robot categories only. A ✓ says a task accepts robots of that category; it does not say that the task is solved, that any learner has been validated on it, or that it has been run on physical hardware. Learner and hardware readiness are tracked separately — see the note below the table.

Task	Manipulator	Mobile Manipulator	Mobile Robot
`lunalab`		✓
`zerog_lab`		✓
`_manipulation`	✓
`debris_capture`	✓
`excavation`
`excavation_demo`
`modular_assembly`	✓
`multi_peg_in_hole`	✓
`multi_sample_collection`	✓
`peg_in_hole`	✓
`sample_collection`	✓
`screwdriving`	✓
`solar_panel_assembly`	✓
`_aerial`			✓
`_ground`			✓
`_orbital`			✓
`aerial_navigation`			✓
`beam_transport`			✓
`flight`			✓
`formation_following`			✓
`formation_following_decentralized`			✓
`formation_following_orbital`			✓
`landing`			✓
`locomotion_velocity_tracking`			✓
`locomotion_waypoint_navigation`			✓
`moonfall`			✓
`orbital_evasion`			✓
`orbital_waypoint_navigation`			✓
`rendezvous`			✓
`terrain_landscaping`
`terrain_landscaping_crater`
`terrain_landscaping_mound`
`traversal`			✓
`velocity_tracking`			✓
`waypoint_navigation`			✓
`_aerial_manipulation`		✓
`_ground_manipulation`		✓
`_orbital_manipulation`		✓
`mobile_debris_capture`		✓
`spacewalk`		✓
`spacewalk_eva`		✓
`spacewalk_iva`		✓

What this table does not say

A row in this matrix is not a support claim for any learner or for physical hardware.

For the terrain landscaping tasks specifically, the measured state is:

	`terrain_landscaping_crater`	`terrain_landscaping`
Dreamer integration runs one environment end to end	yes, with a reliability caveat	not measured
Dreamer learnability demonstrated	no — the gate could not be measured	no
TD-MPC2 supported	no — disabled, fails closed	no — disabled, fails closed
Batched training (`env.num_envs > 1`)	no — refused fail-closed	no
Environment reachability by a scripted reference	no	no
Real-hardware validation	none exists	none exists

Both landscaping rows above are blank in the matrix itself. That is the honest reading: neither task is offered as a solved, learner-validated, or deployable scenario. See Terrain Landscaping Training Readiness for the gate-by-gate detail and docs/superpowers/evidence/terrain_landscaping/readiness.json for the machine-readable report.

Capability evidence ledger

SRB records system capability claims in docs/data/capabilities.toml. The ledger is deliberately conservative: source code proves only that a capability is present. Higher claims require current execution evidence.

Promotion ladder

Every lane uses the same contiguous ladder:

present
unit_verified
simulator_verified
real_data_verified
hardware_verified

A lane cannot skip a level. An unearned level carries a reason, owner, requested evidence, and expiry date. Expired blockers remain visible and do not become evidence.

The ledger has twelve lanes. All twelve are currently present, and none has reached its target. Historical reports are useful context, but they do not promote the current tree.

Evidence identity

All source records include the SHA-256 digest of a repository-relative file. Changing or removing that file invalidates the record.

Execution and benchmark records bind to two identities:

code_sha identifies the repository commit.
scope plus source_sha256 identifies the exact owner-defined input set, including relevant configuration, submodules, data, or dirty-tree state.

The orchestrator computes each current scope digest and passes it to the evaluator. This keeps Git, CI, simulator, data, and hardware orchestration outside the ledger module while preventing results from being combined across different inputs.

Benchmarks additionally record the command, exit code, environment, configuration and artifact digests, failure class, and resource metrics.

Local producer policy

Executable authority is separate from capability status:

docs/data/capabilities.toml owns lane metadata, target levels, blockers, and non-executable producer IDs.
docs/data/capability_producers.toml owns exact argv, working directory, evidence kind, source roots, benchmark configuration roots, timeout, and maximum evidence age.

The collector accepts a producer ID, not a caller-selected command, level, scope, source list, or exit code. It derives the checked-out Git HEAD, hashes the policy-declared scope, executes argv without a shell, captures the real result and logs, then re-hashes the scope. A source mutation, launch failure, timeout, nonzero exit, policy mismatch, future date, or expired record cannot promote a lane.

Producer kind is required and digest-bound. Execution producers cannot declare benchmark configuration inputs. Benchmark producers must declare a sorted, unique configuration-source subset of their source scope, and external record kind must exactly match that policy.

Each record binds the selected producer digest and the complete policy-file digest. Each report binds those digests again alongside the ledger digest. Every retained execution record has the exact attestation local-unattested. Its digests detect mutation and identity mismatch; they do not authenticate who ran it or authorize CI, merge, release, or readiness acceptance.

Local evidence runs

The executable scripts/capability_gates/run_local.bash runner invokes the three approved local end-to-end producers: python-unit-build, terrain-minimal, and python-packaging. It keeps each lane’s logs and record, runs later lanes after an earlier lane fails, and generates one report from the records that exist. A nonzero gate or report result makes the runner nonzero. Each producer retains its behavior gate and then:

loads its local producer policy and computes the declared source scope;
executes the exact gate argv and captures stdout, stderr, timing, and the actual return code;
writes a content-addressed, local-unattested unit_verified execution record even when the gate fails;
re-hashes that scope and evaluates the canonical ledger with the record;
requires the lane to reach unit_verified;
retains logs, evidence, and the report locally before propagating gate or report failure.

The Python record covers its audit, strict typing, unit, terrain-unit, and ownership gates. The terrain-minimal record covers the explicit no-SciPy assertion and core-only face-budget bake. The packaging record covers uv build --offline --no-python-downloads --wheel --sdist --python <selected-host-python>, producing the wheel and source-distribution gate without sync or download. SRB_PYTHON, when set, selects that interpreter; otherwise the gate resolves python3. The explicit build interpreter prevents implicit selection of an inaccessible project .venv, and the same host interpreter creates the disposable validation venv. Artifacts are written beside the evidence under ignored outputs/ storage, never dist/. The record carries a canonical environment string, a configuration digest over pyproject.toml, uv.lock, and the packaging gate, and an artifact digest over a sorted path/size/SHA-256 manifest. It also records finite, nonnegative wall seconds and artifact bytes. After requiring exactly one wheel and one sdist, the gate creates a trap-cleaned disposable venv outside the artifact directory, installs that wheel with offline/no-dependency controls and an explicit venv interpreter, then verifies the installed srb distribution metadata and import srb. The venv is never part of the artifact manifest. Their evidence is retained per lane, then evaluated together by the local report; local records cannot promote another lane or any higher level.

A nominally successful packaging command with missing, empty, unreadable, or non-regular artifacts is retained as a failed benchmark with stable exit code 87 and an artifact-specific failure class. Command failures, timeouts, and source changes also retain schema-valid benchmark records without promotion. A passing schema-v3, local-unattested packaging benchmark has been retained after the interpreter-selection repair. It is scoped local evidence only: the aggregate evaluator still enforces source freshness and every higher-level blocker. A passing standalone report can promote only this local capability; it is not P2, CI, merge, release, or readiness acceptance by itself.

If a scoped source is mutated during a gate, the record uses exit code 86. If a source is deleted or becomes unreadable before the post-run scope can be recomputed, the collector still writes the schema-valid pre-run record with source_changed_during_gate, exit code 86, and concise scope-error metadata. Pre-run scope failures remain fail-closed and produce no evidence record.

The checked-in ledger remains conservative without that external record. Local records cannot promote a different commit or changed source scope. They remain unattested and cannot represent hosted acceptance. GitHub, scheduled, self-hosted, Rust/ROS, network, Isaac/GPU, SPICE, selected-real-data, and hardware attestation is deferred until the repository is public and an authenticated producer contract is approved.

Evaluation seam

srb.core.capability_ledger.evaluate_capability_ledger is the only public evaluation seam:

from srb.core.capability_ledger import evaluate_capability_ledger

report = evaluate_capability_ledger(
    current_code_sha=commit_sha,
    current_source_digests={
        "python-ci": python_ci_scope_sha256,
        "terrain-data": terrain_data_scope_sha256,
    },
    evidence_paths=[python_ci_evidence_path],
)

The evaluator performs no tests or benchmarks. It validates the ledger, checks source hashes, rejects stale or expired execution evidence, enforces contiguous promotion, and returns frozen dataclasses.

Each report carries:

the exact ledger SHA-256 digest;
the local producer-policy path and SHA-256 digest;
the exact local-unattested attestation;
the current commit and sorted source-scope digests;
the canonical digests of all external evidence records;
the exact required capability levels asserted by the report producer;
per-lane effective levels, blockers, and machine-readable issues;
a deterministic report_sha256 over the complete report payload.

Persist the JSON from report.to_json() as local engineering evidence. Its checksum provides integrity, not authentication. A report with zero issues is not CI, merge, release, or readiness acceptance, and it cannot by itself authorize any of those decisions.

Action Space Catalog

Each robot exposes one or more action modes, selected at launch via env.robot.action_mode=<mode> (the default mode is marked *). Dims is the action-vector length and Scale is the multiplier applied to the policy output before it reaches the controller; ? means the value is resolved from the loaded asset at runtime.

Robot	Mode	Controller	Dims	Scale
Canadarm3	`diff_ik` *	Differential Inverse Kinematics	6	0.1
Canadarm3	`joint_pos_rel`	Relative Joint Position	7	1.0
Franka	`diff_ik` *	Differential Inverse Kinematics	6	0.1
Franka	`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
Franka	`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
Franka	`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
Franka	`joint_pos_rel`	Relative Joint Position	7	1.0
Kinova J2N6S	`diff_ik` *	Differential Inverse Kinematics	6	0.1
Kinova J2N6S	`joint_pos_rel`	Relative Joint Position	6	1.0
Kinova J2N7S	`diff_ik` *	Differential Inverse Kinematics	6	0.1
Kinova J2N7S	`joint_pos_rel`	Relative Joint Position	7	1.0
Kinova Gen3N7	`diff_ik` *	Differential Inverse Kinematics	6	0.1
Kinova Gen3N7	`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
Kinova Gen3N7	`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
Kinova Gen3N7	`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
Kinova Gen3N7	`smooth_osc`	Smooth Operational Space Controller	6	pos=0.1, ori=0.1
Kinova Gen3N7	`joint_pos_rel`	Relative Joint Position	7	1.0
Levion Arms	`dual_arm_joint_pos_abs` *	Joint Position To Limits	4	1.0
So Arm100 D5	`diff_ik` *	Differential Inverse Kinematics	6	0.05
So Arm100 D5	`osc_var_stiffness`	Operational Space Controller	12	pos=0.05, ori=0.05, stiffness=120.0
So Arm100 D5	`joint_pos_rel`	Relative Joint Position	5	1.0
So Arm100 D7	`diff_ik` *	Differential Inverse Kinematics	6	0.05
So Arm100 D7	`joint_pos_rel`	Relative Joint Position	7	1.0
Unitree Z1	`diff_ik` *	Differential Inverse Kinematics	6	0.05
Unitree Z1	`joint_pos_rel`	Relative Joint Position	6	1.0
Ur3	`diff_ik` *	Differential Inverse Kinematics	6	0.1
Ur3	`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
Ur3	`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
Ur3	`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
Ur3	`joint_pos_rel`	Relative Joint Position	6	1.0
Ur3E	`diff_ik` *	Differential Inverse Kinematics	6	0.1
Ur3E	`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
Ur3E	`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
Ur3E	`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
Ur3E	`joint_pos_rel`	Relative Joint Position	6	1.0
Ur5	`diff_ik` *	Differential Inverse Kinematics	6	0.1
Ur5	`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
Ur5	`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
Ur5	`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
Ur5	`joint_pos_rel`	Relative Joint Position	6	1.0
Ur5E	`diff_ik` *	Differential Inverse Kinematics	6	0.1
Ur5E	`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
Ur5E	`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
Ur5E	`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
Ur5E	`joint_pos_rel`	Relative Joint Position	6	1.0
Ur10	`diff_ik` *	Differential Inverse Kinematics	6	0.1
Ur10	`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
Ur10	`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
Ur10	`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
Ur10	`joint_pos_rel`	Relative Joint Position	6	1.0
Ur10E	`diff_ik` *	Differential Inverse Kinematics	6	0.1
Ur10E	`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
Ur10E	`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
Ur10E	`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
Ur10E	`joint_pos_rel`	Relative Joint Position	6	1.0
Ur16E	`diff_ik` *	Differential Inverse Kinematics	6	0.1
Ur16E	`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
Ur16E	`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
Ur16E	`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
Ur16E	`joint_pos_rel`	Relative Joint Position	6	1.0
Ur20	`diff_ik` *	Differential Inverse Kinematics	6	0.1
Ur20	`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
Ur20	`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
Ur20	`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
Ur20	`joint_pos_rel`	Relative Joint Position	6	1.0
Ur30	`diff_ik` *	Differential Inverse Kinematics	6	0.1
Ur30	`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
Ur30	`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
Ur30	`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
Ur30	`joint_pos_rel`	Relative Joint Position	6	1.0
Random Ur Manipulator	`diff_ik` *	Differential Inverse Kinematics	6	0.1
Random Ur Manipulator	`osc_fixed`	Operational Space Controller	6	pos=0.1, ori=0.1
Random Ur Manipulator	`osc_var_stiffness`	Operational Space Controller	12	pos=0.1, ori=0.1, stiffness=120.0
Random Ur Manipulator	`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
Random Ur Manipulator	`joint_pos_rel`	Relative Joint Position	6	1.0
Vispa	`diff_ik` *	Differential Inverse Kinematics	6	0.05
Vispa	`osc_var_full`	Operational Space Controller	18	pos=0.1, ori=0.1, stiffness=120.0, damping=1.0
Vispa	`joint_pos_rel`	Relative Joint Position	6	1.0
Zeroglab Arms	`dual_arm_osc` *	Operational Space Controller	24	arm_ceiling: pos=0.1, ori=0.1, stiffness=120.0, arm_wall: pos=0.1, ori=0.1, stiffness=120.0
Humanoid21	`joint_pos` *	Joint Position To Limits	runtime-resolved	1.0
Humanoid28	`joint_pos` *	Joint Position To Limits	runtime-resolved	1.0
Unitree H1	`joint_pos` *	Joint Position To Limits	runtime-resolved	1.0
Unitree G1	`joint_pos` *	Joint Position To Limits	runtime-resolved	1.0
Crazyflie	`body_acc` *	Multicopter Body Acceleration	runtime-resolved	0.5
Ingenuity	`body_acc` *	Multicopter Body Acceleration	2	0.5
Heron	`thrust` *	Thrust	2	1.0
Anymal C	`joint_pos` *	Joint Position To Limits	runtime-resolved	1.0
Anymal D	`joint_pos` *	Joint Position To Limits	runtime-resolved	1.0
Random Anymal Quadruped	`joint_pos` *	Joint Position To Limits	runtime-resolved	1.0
Cadre	`drive` *	Wheeled Drive	runtime-resolved	lin=0.4, ang=0.7853981633974483
Cassie	`joint_pos` *	Joint Position To Limits	12	1.0
Husky	`drive` *	Wheeled Drive	runtime-resolved	lin=0.1, ang=0.7853981633974483
Leo Rover	`drive` *	Wheeled Drive	runtime-resolved	lin=0.4, ang=1.0471975511965976
Mars Rover	`drive` *	Wheeled Drive	runtime-resolved	lin=0.5, ang=0.7853981633974483
Nifty	`nifty` *	Joint Velocity	12	joint_vel_wheel_drive=1.5707963267948966, joint_pos_steer=0.7853981633974483, joint_pos_suspension_yaw=0.5235987755982988, joint_pos_suspension_roll=0.2617993877991494
Open Source Rover	`drive` *	Wheeled Drive	runtime-resolved	lin=0.2, ang=0.3490658503988659
Perseverance	`drive` *	Wheeled Drive	runtime-resolved	lin=0.7, ang=1.3089969389957472
Pragyan	`drive` *	Wheeled Drive	runtime-resolved	lin=0.35, ang=1.3962634015954636
Raph Rover	`drive` *	Wheeled Drive	runtime-resolved	lin=0.4, ang=1.0471975511965976
Raph Rover	`drive_filtered`	Filtered Wheeled Drive	runtime-resolved	lin=0.4, ang=1.0471975511965976
Spot	`joint_pos` *	Joint Position	runtime-resolved	0.2
Unitree A1	`joint_pos` *	Joint Position To Limits	runtime-resolved	1.0
Unitree Go1	`joint_pos` *	Joint Position To Limits	runtime-resolved	1.0
Unitree Go2	`joint_pos` *	Joint Position To Limits	runtime-resolved	1.0
Random Unitree Quadruped	`joint_pos` *	Joint Position To Limits	runtime-resolved	1.0
Floating Platform	`thrust_only` *	Thrust	8	1.0
Floating Platform	`thrust_and_reaction`	Thrust	9	thrust=1.0, reaction_wheel_torque=0.05
Int Ball1	`thrust` *	Thrust	6	1.0
Int Ball2	`thrust` *	Thrust	6	1.0
Apollo Lander	`thrust` *	Thrust	17	1.0
Peregrine Lander	`thrust` *	Thrust	17	1.0
Vikram Lander	`thrust` *	Thrust	13	1.0
Resilience Lander	`thrust` *	Thrust	9	1.0
Random Lander	`thrust` *	Thrust	3	1.0
Iss	`body_acc` *	Body Acceleration	runtime-resolved	0.05
Gateway	`body_acc` *	Body Acceleration	runtime-resolved	0.05
Cubesat	`thrust` *	Thrust	6	1.0
Satellite Mockup	`thrust` *	Thrust	12	1.0
Venus Express	`thrust` *	Thrust	19	1.0
Starship	`thrust` *	Thrust	18	1.0
Super Heavy	`thrust` *	Thrust	59	1.0

Sensor Catalog

Each robot ships pre-configured with one or more sensors. Location is the USD prim path (a regex) where the sensor is attached. Sensor types: Camera (RGB / depth images), Imu (linear acceleration and angular velocity), Contact Sensor (per-link contact state), and Heightmap Sensor (ray-cast elevation grid).

Robot	Sensor Name	Type	Location
Canadarm3	contacts_robot	Contact Sensor	`.*`
Canadarm3	cam_base	Camera	`canadarm3_large_0/camera_base`
Canadarm3	cam_wrist	Camera	`canadarm3_large_7/camera_wrist`
Franka	contacts_robot	Contact Sensor	`.*`
Franka	cam_base	Camera	`panda_link0/camera_base`
Franka	cam_wrist	Camera	`panda_link7/camera_wrist`
Kinova J2N6S	contacts_robot	Contact Sensor	`.*`
Kinova J2N6S	cam_base	Camera	`j2n6s_link_base/camera_base`
Kinova J2N6S	cam_wrist	Camera	`j2n6s_link_6/camera_wrist`
Kinova J2N7S	contacts_robot	Contact Sensor	`.*`
Kinova J2N7S	cam_base	Camera	`j2n7s_link_base/camera_base`
Kinova J2N7S	cam_wrist	Camera	`j2n7s_link_7/camera_wrist`
Kinova Gen3N7	contacts_robot	Contact Sensor	`.*`
Kinova Gen3N7	cam_base	Camera	`base_link/camera_base`
Kinova Gen3N7	cam_wrist	Camera	`end_effector_link/camera_wrist`
So Arm100 D5	contacts_robot	Contact Sensor	`.*`
So Arm100 D5	cam_base	Camera	`Base/camera_base`
So Arm100 D5	cam_wrist	Camera	`Fixed_Gripper/camera_wrist`
So Arm100 D7	contacts_robot	Contact Sensor	`.*`
So Arm100 D7	cam_base	Camera	`Base/camera_base`
So Arm100 D7	cam_wrist	Camera	`End_Servo/camera_wrist`
Unitree Z1	contacts_robot	Contact Sensor	`.*`
Unitree Z1	cam_base	Camera	`link00/camera_base`
Unitree Z1	cam_wrist	Camera	`link06/camera_wrist`
Ur3	contacts_robot	Contact Sensor	`.*`
Ur3	cam_base	Camera	`base_link/camera_base`
Ur3	cam_wrist	Camera	`wrist_3_link/camera_wrist`
Ur3E	contacts_robot	Contact Sensor	`.*`
Ur3E	cam_base	Camera	`base_link/camera_base`
Ur3E	cam_wrist	Camera	`wrist_3_link/camera_wrist`
Ur5	contacts_robot	Contact Sensor	`.*`
Ur5	cam_base	Camera	`base_link/camera_base`
Ur5	cam_wrist	Camera	`wrist_3_link/camera_wrist`
Ur5E	contacts_robot	Contact Sensor	`.*`
Ur5E	cam_base	Camera	`base_link/camera_base`
Ur5E	cam_wrist	Camera	`wrist_3_link/camera_wrist`
Ur10	contacts_robot	Contact Sensor	`.*`
Ur10	cam_base	Camera	`base_link/camera_base`
Ur10	cam_wrist	Camera	`ee_link/camera_wrist`
Ur10E	contacts_robot	Contact Sensor	`.*`
Ur10E	cam_base	Camera	`base_link/camera_base`
Ur10E	cam_wrist	Camera	`wrist_3_link/camera_wrist`
Ur16E	contacts_robot	Contact Sensor	`.*`
Ur16E	cam_base	Camera	`base_link/camera_base`
Ur16E	cam_wrist	Camera	`wrist_3_link/camera_wrist`
Ur20	contacts_robot	Contact Sensor	`.*`
Ur20	cam_base	Camera	`base_link/camera_base`
Ur20	cam_wrist	Camera	`wrist_3_link/camera_wrist`
Ur30	contacts_robot	Contact Sensor	`.*`
Ur30	cam_base	Camera	`base_link/camera_base`
Ur30	cam_wrist	Camera	`wrist_3_link/camera_wrist`
Random Ur Manipulator	contacts_robot	Contact Sensor	`.*`
Random Ur Manipulator	cam_base	Camera	`base_link/camera_base`
Random Ur Manipulator	cam_wrist	Camera	`wrist_3_link/camera_wrist`
Vispa	contacts_robot	Contact Sensor	`.*`
Vispa	cam_base	Camera	`link0/camera_base`
Vispa	cam_wrist	Camera	`link6/camera_wrist`
Humanoid21	imu_robot	Imu	`pelvis`
Humanoid21	contacts_feet	Contact Sensor	`(left\|right)_foot`
Humanoid21	cam_front	Camera	`pelvis/camera_front`
Humanoid28	imu_robot	Imu	`pelvis`
Humanoid28	contacts_feet	Contact Sensor	`(left\|right)_foot`
Humanoid28	cam_front	Camera	`pelvis/camera_front`
Unitree H1	imu_robot	Imu	`pelvis`
Unitree H1	contacts_feet	Contact Sensor	`(left\|right)_ankle_link`
Unitree H1	cam_front	Camera	`pelvis/camera_front`
Unitree G1	imu_robot	Imu	`pelvis`
Unitree G1	contacts_feet	Contact Sensor	`(left\|right)_ankle_roll_link`
Unitree G1	cam_front	Camera	`pelvis/camera_front`
Crazyflie	imu_robot	Imu	`body`
Crazyflie	cam_onboard	Camera	`body/camera_onboard`
Ingenuity	imu_robot	Imu	`body`
Ingenuity	cam_onboard	Camera	`body/camera_onboard`
Ingenuity	altimeter	Heightmap Sensor	`body`
Heron	imu_robot	Imu	`base_link`
Heron	cam_onboard	Camera	`base_link/camera_onboard`
Anymal C	imu_robot	Imu	`base`
Anymal C	contacts_feet	Contact Sensor	`.*FOOT`
Anymal C	cam_front	Camera	`base/camera_front`
Anymal D	imu_robot	Imu	`base`
Anymal D	contacts_feet	Contact Sensor	`.*FOOT`
Anymal D	cam_front	Camera	`base/camera_front`
Random Anymal Quadruped	imu_robot	Imu	`base`
Random Anymal Quadruped	contacts_feet	Contact Sensor	`.*FOOT`
Random Anymal Quadruped	cam_front	Camera	`base/camera_front`
Cadre	imu_robot	Imu	`chassis`
Cadre	contacts_wheels	Contact Sensor	`.wheel.`
Cadre	cam_front	Camera	`chassis/camera_front`
Cassie	imu_robot	Imu	`pelvis`
Cassie	contacts_feet	Contact Sensor	`(left\|right)_toe`
Cassie	cam_front	Camera	`pelvis/camera_front`
Husky	imu_robot	Imu	`chassis`
Husky	contacts_wheels	Contact Sensor	`.wheel.`
Husky	cam_front	Camera	`chassis/camera_front`
Leo Rover	imu_robot	Imu	`chassis`
Leo Rover	contacts_wheels	Contact Sensor	`.wheel.`
Leo Rover	cam_front	Camera	`chassis/camera_front`
Mars Rover	imu_robot	Imu	`Body`
Mars Rover	contacts_wheels	Contact Sensor	`.[Dd]rive.`
Mars Rover	cam_front	Camera	`Body/camera_front`
Nifty	imu_robot	Imu	`chassis`
Nifty	contacts_wheels	Contact Sensor	`.wheel.`
Nifty	cam_front	Camera	`chassis`
Open Source Rover	imu_robot	Imu	`chassis`
Open Source Rover	contacts_wheels	Contact Sensor	`.wheel.`
Open Source Rover	cam_front	Camera	`chassis/camera_front`
Perseverance	imu_robot	Imu	`body`
Perseverance	contacts_wheels	Contact Sensor	`.wheel.`
Perseverance	cam_front	Camera	`body/camera_front`
Pragyan	imu_robot	Imu	`body`
Pragyan	contacts_wheels	Contact Sensor	`.wheel.`
Pragyan	cam_front	Camera	`body/camera_front`
Raph Rover	imu_robot	Imu	`chassis`
Raph Rover	contacts_wheels	Contact Sensor	`wheel.*`
Raph Rover	cam_front	Camera	`chassis/camera_front`
Spot	imu_robot	Imu	`body`
Spot	contacts_feet	Contact Sensor	`.*foot`
Spot	cam_front	Camera	`body/camera_front`
Unitree A1	imu_robot	Imu	`trunk`
Unitree A1	contacts_feet	Contact Sensor	`.*foot`
Unitree A1	cam_front	Camera	`trunk/camera_front`
Unitree Go1	imu_robot	Imu	`trunk`
Unitree Go1	contacts_feet	Contact Sensor	`.*foot`
Unitree Go1	cam_front	Camera	`trunk/camera_front`
Unitree Go2	imu_robot	Imu	`base`
Unitree Go2	contacts_feet	Contact Sensor	`.*foot`
Unitree Go2	cam_front	Camera	`base/camera_front`
Random Unitree Quadruped	imu_robot	Imu	`trunk`
Random Unitree Quadruped	contacts_feet	Contact Sensor	`.*foot`
Random Unitree Quadruped	cam_front	Camera	`trunk/camera_front`
Floating Platform	imu_robot	Imu	`base_link`
Floating Platform	cam_onboard	Camera	`base_link/camera_onboard`
Int Ball1	imu_robot	Imu	`base_link`
Int Ball1	cam_onboard	Camera	`base_link/camera_onboard`
Int Ball2	imu_robot	Imu	`base_link`
Int Ball2	cam_onboard	Camera	`base_link/camera_onboard`
Apollo Lander	imu_robot	Imu	`base`
Apollo Lander	cam_onboard	Camera	`base/camera_onboard`
Peregrine Lander	imu_robot	Imu	`base`
Peregrine Lander	cam_onboard	Camera	`base/camera_onboard`
Vikram Lander	imu_robot	Imu	`base`
Vikram Lander	cam_onboard	Camera	`base/camera_onboard`
Resilience Lander	imu_robot	Imu	`base`
Resilience Lander	cam_onboard	Camera	`base/camera_onboard`
Random Lander	imu_robot	Imu	`base`
Random Lander	cam_onboard	Camera	`base/camera_onboard`
Iss	imu_robot	Imu	`base`
Iss	cam_onboard	Camera	`base/camera_onboard`
Gateway	imu_robot	Imu	`gateway`
Gateway	cam_onboard	Camera	`gateway/camera_onboard`
Cubesat	imu_robot	Imu	`cubesat`
Cubesat	cam_onboard	Camera	`cubesat/camera_onboard`
Satellite Mockup	imu_robot	Imu	`satellite_mockup`
Satellite Mockup	cam_onboard	Camera	`satellite_mockup/camera_onboard`
Venus Express	imu_robot	Imu	`base`
Venus Express	cam_onboard	Camera	`base/camera_onboard`
Starship	imu_robot	Imu	`base`
Starship	cam_onboard	Camera	`base/camera_onboard`
Super Heavy	imu_robot	Imu	`base`
Super Heavy	cam_onboard	Camera	`base/camera_onboard`

Stats Dashboard

	Count
Robots	64
- Manipulator	21
- Mobile Manipulator	7
- Mobile Robot	36
Objects	56
Scenery	20
Tasks	42
- Digital Twin	2
- Manipulation	11
- Mobile	22
- Mobile Manipulation	7
Action Modes (total)	121
Sensor Types	4

IDE Configuration

Just like any other software project, Space Robotics Bench development benefits from proper IDE setup.

VSCode Setup

Visual Studio Code (VS Code) is the recommended IDE for working with the Space Robotics Bench codebase. However, you are welcome to use any IDE of your choice.

Extensions

The Python source code of SRB is fully typed, and the recommended extensions for VSCode include:

code --reuse-window \
     --install-extension ms-python.python \
     --install-extension ms-python.vscode-pylance \
     --install-extension ms-python.mypy-type-checker \
     --install-extension charliermarsh.ruff \
     --install-extension tamasfe.even-better-toml

For Rust development, the recommended extensions include:

code --reuse-window \
     --install-extension rust-lang.rust-analyzer \
     --install-extension tamasfe.even-better-toml

Workspace Settings

SRB comes with workspace settings that primarily configure Python paths for Isaac Sim/Lab modules. It does so with these assumptions (which you can adjust as needed):

Isaac Sim is installed at ../isaac-sim
Isaac Lab is installed at ../isaaclab

Using the Dev Container

For a pre-configured development environment, consider using Dev Container, which automatically includes all necessary tools and extensions.

Dev Container

Space Robotics Bench includes a Dev Container configuration via devcontainer.json that you can customize for your development needs.

Prerequisites

Before opening SRB in a Dev Container, make sure your host has:

Docker Engine installed and working
NVIDIA Container Toolkit installed if you want GPU-backed Isaac Sim
xauth installed if you want Isaac Sim GUI forwarding
an X11 session available when you want Isaac Sim GUI forwarding
Visual Studio Code (VS Code)
the Dev Containers extension
the code CLI available on your host if you want to use the helper script below

The Dev Container uses the repository Dockerfile, mounts your local checkout into /root/ws, forwards X11, and reuses host-side Omniverse / shader caches to avoid rebuilding them every time.

Open in VS Code

Visual Studio Code (VS Code) users can open the repository as a Dev Container with the Dev Containers extension:

Open the repository folder in VS Code:
```
code /path/to/space_robotics_bench
```
Run Dev Containers: Reopen in Container from the Command Palette (Ctrl+Shift+P).

VS Code will build the image (if needed) and attach to the container automatically. First build can take a while because it builds the full SRB image.

If you have the VS Code CLI installed, the repository also provides a shortcut:

./.devcontainer/open.bash

Verify Container Setup

Once attached, verify core pieces before starting heavier workflows:

nvidia-smi
srb --help
srb cache update
echo "$DISPLAY"

If you only need terminal access to an already-running SRB container, .docker/join.bash is lighter than opening a second VS Code window.

Testing

Automated testing is an essential part of the development process to ensure the correctness and reliability of the Space Robotics Bench.

Scope

The SRB test suite focuses on integration tests of various workflows that are applied sequentially to all registered environments. As these tests require Isaac Sim to be running, NVIDIA GPU must be available on the system. For this reason, the full integration suite is not run automatically via CI/CD pipelines and must be executed manually. The pure-Python subsets (tests/unit, tests/terrain/unit, tests/test_checkpoint_discovery.py) are run in CI via .github/workflows/python.yml (on every PR and on non-docs pushes to main).

Hint: The fast unit-test subset under tests/unit/ does not require Isaac Sim or a GPU — run it anywhere with pytest tests/unit -q.

`srb test` — Run Tests

The srb test command simplifies running the test suites of SRB via pytest and/or cargo test.

srb test [options]

Options

Argument	Description	Default
`-l`/`--language`/`--lang`	Language suites to test	`[python]`

Hint: This command supports forwarding of all arguments following --.

Examples

Forward -vx arguments to Python test suite (pytest):

srb test -- -vx

Run tests for both Python and Rust:

srb test --lang python rust

Documentation

This documentation is built using mdBook, which creates modern online books from a collection of Markdown files.

`srb doc` (alias `srb docs`) — Local Preview

The srb doc (alias srb docs) command groups the documentation utilities. Without a sub-subcommand it falls back to serve, which is the most common entry point during day-to-day development.

Sub-subcommand	Description
`srb doc serve` (default)	Run an mdBook live-reload server (default action)
`srb doc build`	Build the static documentation site (used by CI/CD)
`srb doc docgen`	Regenerate auto-introspected pages (assets, tasks, catalogs) — launches Isaac Sim headless
`srb doc thumbnail`	Render asset thumbnails into `docs/src/_graphics/assets/` or task thumbnails into `docs/src/_graphics/tasks/`

srb doc                # equivalent to `srb doc serve`
srb doc serve
srb doc build
srb doc docgen --static-only
srb doc thumbnail --asset-type robot
srb doc thumbnail --task excavation_demo
srb doc thumbnail --task ALL

docgen regenerates the asset, task, compatibility, action-space, sensor, stats, and attribution pages, so it launches Isaac Sim headless for introspection. --assets-only and --tasks-only scope which pages are written. --static-only is different: it validates the sidecar inputs and prints the asset/task counts without launching anything or writing any page, which makes it the check to run where Isaac Sim is unavailable. Hand-written sections stay intact between runs when they are wrapped in  …  markers. thumbnail accepts --width, --height, --output-dir, --asset, --asset-type, --task, --force, and --allow-failures (keep a task batch going even when some tasks fail).

Hint: srb doc serve and srb doc build forward any tokens after -- directly to mdbook.

Automated Deployment

The documentation is automatically deployed to GitHub Pages via GitHub Actions. The deployment process is triggered by pushing to the main branch.

Contributing

Documentation is usually the weakest link in most open-source projects. We would greatly appreciate your help in improving this documentation. If you find any errors or have suggestions for improvements, don’t hesitate to open an issue or a pull request. Thank you in advance!

Development Utilities

The Space Robotics Bench provides various utilities to streamline the development workflow and maintain the project.

Update Assets
Clean Cache

Development Utilities - Update Assets

SRB keeps two git-backed repositories for static content:

assets/srb_assets for asset payloads.
docs/src/_graphics for docs/task thumbnails and visuals.

By default, this script updates assets/srb_assets:

./assets/update.bash

You can also bootstrap from a Hugging Face mirror when submodules are not available (private mirror support requires auth via HF_TOKEN or hf auth login):

./assets/update.bash --source hf --repo-id <owner>/<repo>

Optional overrides:

./assets/update.bash --source hf --repo-id <owner>/<repo> --repo-type dataset --revision main --path /path/to/assets/srb_assets

To initialize the graphics repo from Hugging Face:

./assets/update.bash --source hf --repo-id <owner>/<repo> --repo-type dataset --revision main --path /path/to/docs/src/_graphics

--repo-type defaults to dataset; supported values are model, dataset, and space.

For private mirrors, you can also use environment variables.

export SRB_ASSETS_SOURCE=hf
export SRB_ASSETS_HF_REPO_ID=<owner>/<private-assets-repo-or-dataset>
export SRB_ASSETS_HF_REPO_TYPE=dataset   # or model / space
export SRB_ASSETS_HF_REPO_REVISION=main         # optional

./assets/update.bash

./.docker/build.bash also accepts:

SRB_ASSETS_... for assets/srb_assets
SRB_GRAPHICS_... for docs/src/_graphics

Those environment variables are used when a directory is missing and needs bootstrap.

Development Utilities — Clean Cache

The srb clean command removes cached files associated with SRB itself, Isaac Sim, and SimForge. This is the recommended way to recover from stale caches after framework upgrades or asset changes.

SRB and Isaac cache directories must first carry a versioned ownership marker bound to their canonical path, device, and inode. Claim the fixed cache roots explicitly:

srb cache claim-clean-root --target srb
srb cache claim-clean-root --target isaacsim

The fixed repository-local .cache and the listed Isaac cache paths may adopt existing content. A populated custom SRB_CACHE_ROOT is never adopted. Create and claim a new empty custom root before writing cache data instead.

Usage

srb clean [TARGET ...] [-y/--yes]

Argument	Description	Default
`TARGET ...`	One or more of `all`, `srb`, `isaacsim`, `simforge` (multiple allowed)	`[all]`
`-y`/`--yes`	Skip the interactive confirmation prompt	`False`

Selecting all (the default) is equivalent to specifying the srb, isaacsim, and simforge targets. Trailing tokens after -- are forwarded to simforge clean when the simforge target is included.

--yes bypasses only the prompt. It never bypasses ownership, root-depth, repository/home, mount, symlink, device, or entry-identity checks. Removal uses anchored directory descriptors and preserves the cache root and its marker.

What Gets Removed

Target	Affected paths
`srb`	`<repository root>/.cache` (registry caches, completion caches, …)
`isaacsim`	`~/.cache/isaac-sim`, `~/.cache/ov`, `~/.cache/nvidia/GLCache`, `~/.cache/packman`, `~/.nv/ComputeCache`
`simforge`	Delegated to `simforge clean` (procedurally generated SimForge assets)

The delegated SimForge command retains SimForge’s own ownership policy. SRB does not recursively remove its directory.

Examples

Remove all caches with confirmation:

srb clean

Remove only SRB caches without prompting:

srb clean srb -y

Remove only the Isaac Sim Omniverse caches:

srb clean isaacsim -y

Forward extra arguments to SimForge:

srb clean simforge -y -- --asset-name my_asset

Reference: SimForge CLI — simforge clean

Real Terrain Assets

srb.assets.scenery.RealTerrain is the entry point for using real Lunar / Martian terrain — actual Digital Elevation Models (DEMs) from NASA/USGS/ESA surveys, baked into simulation-ready meshes — in place of procedurally generated surfaces. Use it when you need geographically accurate, photorealistic scenes. This guide covers the runtime classes; for adding new DEM sources and the offline DEM → mesh → USD bake pipeline, see Adding Terrain Sites.

Basic usage

from srb.assets.scenery import RealTerrain
from srb.assets.scenery.real_terrain_sites import LunarSiteApollo17, MartianSiteJezero

# Generic constructor — body + site name from manifest.
terrain = RealTerrain(body="moon", site="apollo17")

# Or use the per-site shortcut class (auto-generated from manifest).
terrain = LunarSiteApollo17()

# Or an arbitrary lat/lon region — no manifest entry at all (v2 stacks;
# see the terrain-stacks guide for identity/normalization/fallback rules).
terrain = RealTerrain(body="moon", lat_deg=-89.66, lon_deg=0.0, size_m=2000.0)

# Returns an Isaac Lab AssetBaseCfg you can attach to a scene.
env_cfg.scene.terrain = terrain.asset_cfg

Companion assets

Each RealTerrain exposes optional sky / lighting / rocks companions, all configured through the manifest with per-instance overrides:

# Use everything the manifest specifies for this site (recommended).
terrain = LunarSiteApollo17()
terrain.attach_companions(env_cfg)
# → env_cfg.scene now has skydome, lighting, rocks_00, ... attached.

# Override individual companions.
terrain = LunarSiteApollo17(
    sky="apollo17_pan",      # explicit catalog name
    lighting="auto",         # use manifest default (default value)
    rocks=None,              # disable rocks
)

The env path now attaches companions automatically (OI-1, terrain v2 Phase 4): when env.scenery resolves to a RealTerrain with attach_companions_enabled=True (the default), BaseEnvCfg._add_scenery calls attach_companions itself — arbitrated against the env.ephemeris sun/skydome drives, with an all-or-nothing warn-once degrade when companion products cannot be built (the shipped sites’ companion catalog entries are placeholders until operator action O3, so today every curated site degrades and renders as before). Set attach_companions_enabled=False to opt a task out. The precedence table and the rendered-output caveat live in Terrain Stacks → Companions and the ephemeris sun.

Ephemeris-driven lighting

A site whose manifest sets lighting: ephemeris (plus coords + a curated epoch) bakes its Sun / Earthshine lights from real SPICE ephemeris at that epoch rather than a static preset, via srb/terrain/lights/ephemeris_bridge.py. The bridge runs one query(), folds the terrain’s persisted georef.north_convergence_deg into the sun azimuth (az_grid = az_true + γ), and writes an epoch-keyed, content-hashed lights.usd (.../srb_lights/ephemeris/<body>_<site>_<epoch>_<hash8>/) so a changed epoch or grid-north correction yields a fresh file. With the pinned NAIF kernels absent it warns once and falls back to the body-default static preset (air-gapped-safe); srb ephemeris download enables the real path. See Ephemeris → Terrain lighting for the manifest fields, the offline fallback, and the srb asset list provenance nuance (listed azimuth is raw true-north; the bake folds +γ).

SimForge augmentation

For per-env variation (rocks scattered with a different seed per env), pass augment="simforge":

terrain = RealTerrain(
    body="moon", site="apollo17", augment="simforge",
)
# Baked terrain is the static backbone; SimForge generates rocks on top.
# Augment params come from `augment_preset="lunar_default"` (default).

augment="simforge" requires simforge_foundry to be importable; otherwise raises SimforgeMissingError.

User-supplied DEMs

Skip the manifest entirely and bake a local DEM:

terrain = RealTerrain(
    body="moon",
    dem_path="/data/my_lunar_dem.tif",
    site_name="my_polar_crater",      # cache key
    attribution="Acme Surveying, 2026",
    patch_size_m=300,
    use_ortho=False,
)

attribution is REQUIRED for user-supplied DEMs — license compliance.

site, dem_path, and the region triplet (lat_deg/lon_deg/size_m) are mutually exclusive selection modes — exactly one must be given, and setting two (including site + dem_path together, which used to slip through) is rejected at the spec boundary. Note the per-site shortcut classes bake site in as a class default, so LunarSiteApollo17(dem_path=...) is a both-set error unless you also explicitly clear site=None — for a local DEM, plain RealTerrain(body=..., dem_path=...) is the clearer spelling.

CLI workflow

# Discover available sites + presets.
srb asset list terrain
srb asset list terrain --body moon
srb asset list rocks
srb asset list hdri
srb asset list lights
srb asset info terrain apollo17

# Pre-bake explicitly (otherwise it bakes lazily on first use).
srb asset download terrain apollo17
srb asset download terrain --all --body moon
# Arbitrary region (v2 stacks; no manifest site — mutually exclusive with a
# positional target/--all and the site-only flags):
srb asset download terrain --lat -89.66 --lon 0.0 --size-m 2000 \
                           --body moon --deshade
srb asset bake terrain --dem /data/my.tif --body moon --site-name my_local \
                       --attribution "Me, 2026"

# Manage the cache.
srb asset cache-stats
srb asset clean terrain apollo17 --baked
srb asset clean terrain --all
srb asset validate-manifest

# CI: warm everything at lod=low.
srb asset prefetch-fixtures

Real-terrain v2: source catalog and stack resolution

A per-body source catalog (srb/terrain/catalog/<body>.yaml, schema in srb/terrain/catalog_schema.py) and a resolver (srb.terrain.catalog.resolve.resolve_stacks) exist alongside the v1 manifest described above. Given a location, the resolver turns the catalog into a CompositionPlan: a prioritized DEM stack, plus sunlit-color, PSR-color, mask, and quality stacks, each layer’s estimated fetch bytes, and which layers would need large-download consent. The v2 resolver and bake path are live. The srb asset stack command is only a read-only preview: it never touches the network or moves a byte.

srb asset stack --body moon --lat -89.46 --lon 137.3 --size 2000
srb asset stack --body moon --lat -89.46 --lon 137.3 --size 2000 --json

The per-body source catalogs now carry real, curated data — but coverage is narrow. srb/terrain/catalog/moon.yaml carries 19 real, sha256-pinned layers; mars.yaml and earth.yaml carry 2 each (one HiRISE DTM+ortho pair for Jezero; one Copernicus polar tile and one 3DEP non-polar tile). For access: cog_remote remains a catalog compatibility label. Production first downloads the complete source through SRB’s redirect-validating, size-bounded provider, verifies the pinned SHA-256, then reads local raster windows. Direct GDAL /vsicurl network access is refused because it cannot enforce SRB’s per-hop destination-host policy. First use therefore needs full-source storage and large-download consent where applicable; content-addressed reuse avoids a second download. Each contributing layer records bytes_verified: true in its meta.json fusion provenance. Curated layer data (a real, downloaded-once sha256 per file, a reviewed footprint, an assigned priority_class) is added by a human, offline (operator action O3; see Adding a v2 source layer) — this is an ongoing, incremental process, not a one-shot completed catalog. A request whose footprint falls outside every curated layer still resolves to an empty plan, and srb asset stack prints the same v1-fallback line RealTerrain itself falls back to: no v2 source coverage for <body> at (<lat>, <lon>) — falling back to the v1 single-source path. See Terrain Stacks → Catalog status per body for the exact curated layer list.

Wired for both request forms. A srb/terrain/manifest.yaml site whose sources declares a dem_stack (and, optionally, a color_stack) under a version: 2 manifest bakes end to end through the exact same RealTerrain(body=..., site=...) / srb asset download terrain <site> / srb asset bake calls a v1 site uses — bake_or_load recognizes it automatically (sources.dem is None on a v2 site) and runs it through the fusion engine described below. See Adding a v2 site with a declared stack for the full walkthrough. An arbitrary region — RealTerrain(body="moon", lat_deg=..., lon_deg=..., size_m=...) with no manifest site/patch at all, or the matching srb asset download terrain --lat --lon --size-m CLI form — resolves its stacks by footprint intersection and bakes through the same path, with a deterministic slug identity, degrade-to-procedural in envs, and an illumination-correction (deshade) knob. That request path has its own operator guide: Terrain Stacks — identity/normalization rules, the body-aware (polar canonical / non-polar local-stereographic) resolution CRS, deshade precedence and cache-keying, and the companion/ephemeris arbitration table all live there.

Consent — read this before trusting the preview. srb asset stack’s “consent required” line is a descriptive preview (CompositionPlan.consented, an OR of --allow-large and SRB_TERRAIN_LARGE_OK) — it is not what actually protects a fetch. The real, non-bypassable gate (check_layer_consent / layer_max_bytes in srb/terrain/sources/acquisition.py, an AND of the catalog layer’s own allow_large_download flag and SRB_TERRAIN_LARGE_OK) lives on the fetch path and re-derives consent independently, per layer — it never reads the preview’s consented field. Seeing consent required: none in srb asset stack’s output is not authorization to fetch anything; nothing in Phase 2 fetches automatically either way (no code path calls input() or prompts — consent is env-var/flag only, never interactive).

Real-terrain v2: the fusion engine

A v2 site’s dem_stack names one role: base layer (gap-free, covers the whole patch) plus zero or more role: inset layers (finer, partial coverage); color_stack is the same shape for colour. srb/terrain/fusion/ composes them into one DEM and one colour raster:

Every contributing layer is warped onto the patch’s own grid (fusion.grid.warp_to_grid).
Each DEM inset is co-registered against the base (published catalog shift, or an estimated one when the catalog records none) and demoted — dropped, with a warning — if the QC residual is too large.
A feathered priority blend composites the DEM stack: the finest/ highest-priority inset wins its own interior outright, feathering to the next layer only within a narrow band (blend_len_m, clamped to 50–200 m) of its own true footprint boundary — never a global weighted average, and never the layer’s declared footprint, only where it actually has data.
Colour is composited the same way (first-finite-wins over the sorted stack) and radiometrically normalized against the stack’s anchor layer.

What a fused bake adds to the artifact, alongside the same terrain.usd / textures a v1 bake produces:

provenance.tif — a single-band uint8 GeoTIFF the same size as the fused DEM: 0 means no layer contributed there, 1..N is the 1-based index of the layer that won that pixel (matching meta["fusion"]’s own per-layer index).
meta.json["fusion"] — a new top-level block (v1 bakes never carry it; meta["georef"] is unrelated and unchanged in shape either way): fusion_version, grid (crs, center_xy_m, size_m, gsd_m, pixels), dem_layers / color_layers (each entry: layer_id, role, kind, index, area_fraction, native_gsd_m, upsampled, radiometry, alignment, license, attribution), contributors (loaded mask inputs that affected colour selection but own no raster index), provenance_tif / uncertainty_tif (filenames or null), and warnings.
meta.json["bake_identity"] — the versioned cache identity for a fused generation: cache key, canonical DEM and colour/auxiliary stack hashes, fusion version, target GSD, and effective deshade state. A fused generation without this record is not ready.
ATTRIBUTION.txt — one layer_id / license / attribution block per contributing layer (base, every surviving inset, and loaded mask contributor), in blend order — the v1 form (a single attribution + "\n" string) only ever applies to a v1 (single-source) bake.

Budgets — v2 only; v1 is unchanged forever. A v1 bake keeps its stride-based LOD mesh and full-resolution, uncapped textures exactly as before. A v2 (fused) bake additionally quadric-decimates the mesh toward a per-LOD face budget and caps texture/AO resolution (srb/terrain/fusion/budgets.py):

LOD	Max mesh faces (grid)	Max texture side (px)
`high`	4,000,000	8192
`medium`	1,000,000	4096
`low`	250,000	2048

Ambient occlusion is computed at ≤2048 px regardless of LOD, then upsampled. The face budget applies to the decimated grid; the (never decimated) skirt and solid-shell geometry appended for collision adds a small, perimeter- scaling number of faces on top — the shipped mesh can land a little over the nominal number for that reason, never because the budget failed to bite.

This path is proven against real data for region requests; no declared- stack manifest site uses it yet. The fusion engine, the v2 request plumbing, and the budgets are all real, tested, and reachable both offline (a hand-built manifest + fixture catalog exercises the whole path; see tests/terrain/integration/test_fusion_end_to_end.py) and, since the SPICE/Terrain remaining-work plan, against real network products: four real region bakes (moon polar, mars Jezero, earth polar, earth non-polar) complete end to end through this exact fusion path, area_fraction=1.0 on every base layer (see Terrain Stacks → Catalog status per body and docs/research/real_terrain_v2/). What remains not done: manifest.yaml is still version: 1 — no site declares a dem_stack/color_stack against the real catalog (see Adding a v2 site with a declared stack — its connecting_ridge example is still illustrative, using layer ids that don’t exist in the shipped catalog), so a declared-stack v2 site remains untried; only the arbitrary-region request form has been exercised against real data.

Caching and reproducibility

Bakes are content-addressed by (body, site, patch, lod, seed, materials_version, dem_sha256, ...). The same inputs always produce the same output bytes; changing any input (e.g., bumping materials_version) invalidates the bake.

Cache root: assets/srb_assets/scenery/terrain/<body>/<site>/<patch_key>/. Override via SRB_TERRAIN_CACHE_ROOT=/custom/path.

Disabling auto-fetch

By default, missing bakes are produced on first use. To require an explicit pre-bake step (e.g., air-gapped CI):

export SRB_TERRAIN_AUTO_FETCH=0

RealTerrain(...) will then raise TerrainNotBakedError with the exact srb asset download command to run.

Troubleshooting

Symptom	Likely cause	Remediation
`ManifestSiteUnknownError`	typo in `site=`	`srb asset list terrain`
`LicenseMissingError`	user DEM without `attribution`	pass `attribution="..."`
`ChecksumMismatchError`	upstream DEM changed	`srb asset validate-manifest --check-shas`
`SimforgeMissingError`	`augment="simforge"` w/o package	`pip install simforge_foundry`
`TerrainNotBakedError`	`auto_fetch=False` + cache miss	run the `srb asset download` shown

Units & Conventions

This page documents the unit system and conventions used throughout the Space Robotics Bench.

Unit System

SRB uses SI units consistently, following the conventions of NVIDIA Isaac Sim and Isaac Lab.

Quantity	Unit	Symbol
Length	meter	m
Mass	kilogram	kg
Time	second	s
Angle	radian	rad
Force	newton	N
Torque	newton-meter	N·m
Gravity	meters per second squared	m/s²

Coordinate Frame

SRB uses a right-handed coordinate system with Z-up convention:

X — forward
Y — left
Z — up
Gravity — applied in the negative Z direction

Quaternion Format

Quaternions follow the (x, y, z, w) convention throughout SRB and Isaac Lab. This is important to keep in mind when interfacing with external libraries that may use the (w, x, y, z) order.

Note: Some USD/OpenUSD APIs (e.g., Gf.Quatf) use the (w, x, y, z) order internally. SRB handles the conversion automatically when interacting with these APIs.

Frame Suffix Convention

State variables use suffixes to indicate the reference frame:

Suffix	Reference Frame	Example
`_w`	World frame (absolute)	`root_pos_w`, `body_quat_w`
`_b`	Body/base frame (relative)	`ee_vel_b`

Domain-Specific Gravity

Each simulation domain defines its own gravitational acceleration:

Domain	Gravity (m/s²)
Earth	9.80665
Mars	3.72076
Moon	1.62496
Asteroid	0.14219
Orbit	0.0

Spec Authoring (`@spec` decorator)

New ActionGroup, SensorBase, and observation-manager classes should declare their spec via the @spec decorator from srb.core.spec. The decorator registers a ComponentDescriptor at import time; the env aggregates these into a queryable EnvSpec accessible via env.srb_spec (distinct from gymnasium’s own env.spec slot).

Two authoring styles

Explicit fields= — best for short, fixed schemas:

from srb.core.spec import spec, TermSpec

@spec(kind="action", fields=[
    TermSpec(name="lin", shape=(1,), dtype="float32",
             units="m/s", min=-2.0, max=2.0),
    TermSpec(name="ang", shape=(1,), dtype="float32",
             units="rad/s", min=-1.0, max=1.0),
])
class WheeledDriveActionGroup(ActionGroup):
    ...

spec_field attributes — best for longer schemas (>5 fields) where field-by-field readability matters:

from srb.core.spec import spec, spec_field

@spec(kind="action")
class SmoothOSCActionGroup(ActionGroup):
    ee_pos = spec_field("ee_pos", dtype="float32", shape=(3,), units="m")
    ee_rot = spec_field("ee_rot", dtype="float32", shape=(4,), units="quat")
    stiffness = spec_field("stiffness", dtype="float32", shape=(6,),
                           units="N/m, Nm/rad")

Picking one or the other is mandatory — passing both fields= and using spec_field() in the same class raises at decoration.

What `kind=` to choose

"action" — ActionGroup subclasses
"sensor" — SensorBase subclasses
"observation" — observation-manager classes (currently per-task; pass group="policy" as a keyword extra to scope to a specific group)

Privileged terms

Mark observation terms that must NOT appear in deployed policies (e.g. ground-truth particle positions) with privileged=True:

TermSpec(name="particles_xyz", shape=(N, 3), dtype="float32",
         privileged=True,
         semantic_label="excavation.particle_positions")

projections.onnx_export_mask(spec.observation) returns the set of term names to filter from ONNX export: every term with privileged=True plus any low-trust term whose provenance starts with <fallback> — pass this mask to the ONNX exporter to filter them out.

CI gate

A regression test in tests/unit/test_spec_decorator_audit.py enforces that every concrete ActionGroup and SRB-native SensorBase subclass is registered. Adding a new subclass without @spec(...) fails the test.

Contributing — New Assets

This guide explains how to contribute new assets to the Space Robotics Bench.

Asset Types Overview

The Space Robotics Bench categorizes assets into three main types:

Asset Type	Description
Scenery	Environmental elements like terrain and structures
Objects	Interactive objects, tools, and payloads
Robots	Systems that are intended to be controlled

Static Assets

All static assets used by the Space Robotics Bench are separated into the srb_assets repository to encourage their reuse.

If you wish to contribute your asset, please follow these guidelines:

Simplify the mesh for efficient simulation
Aim for watertight meshes with clean topology
Bake materials into PBR textures for compatibility
Export meshes in the USD format (.usd/.usda/.usdc/.usdz)
For articulated assets, add relevant joints and APIs (you can use Isaac Sim for this)

Afterward, you can add the asset to your fork of srb_assets and submit a pull request. We greatly appreciate your contributions!

Procedural Assets with SimForge

Reference: SimForge

SimForge is a framework for creating diverse virtual environments through procedural generation. SRB leverages SimForge to generate all procedural assets. Below are some examples:

Please refer to the SimForge documentation if you wish to contribute new procedural assets, particularly the following sections:

New Assets
- Blender Generator: New Assets

We look forward to seeing your fantastic contributions. Many thanks in advance!

Contributing — New Tasks

The process of introducing a new environment into the Space Robotics Bench is intended to be straightforward, with a limited amount of boilerplate that you need to write yourself. This guide will walk you through the primary steps for creating a new SRB task.

1. Duplicate a Template

Navigate to the srb/tasks directory in your local repository. Then, duplicate one of the existing task templates and rename it to your desired task name. It is recommended that you keep your task in the same root directory as the template in order to simplify the registration process. You should select a template based on the type of task and scenario you wish to create:

Template	Description
`_manipulation`	Fixed-base manipulation with robotic arms
`_ground`	Ground traversal on planetary surfaces
`_aerial`	Aerial navigation above planetary surfaces
`_orbital`	Spaceflight maneuvers
`_ground_manipulation`	Mobile manipulation with ground-based robots
`_aerial_manipulation`	Mobile manipulation with flying robots
`_orbital_manipulation`	Mobile manipulation with spacecraft

2. Modify the Environment

Now, it is time to be creative and modify the environment to suit your needs. You can adjust the following components:

Scene: Change the assets and layout of the environment to match your scenario.
Robot: Specify a category of robots that are suitable for performing the task.
Task: Define the task-specific logic and objectives.
- Observation Space: Define the observation space for the task.
- Reward Function: Define the reward function for the task.
- Termination Condition: Define the termination condition for the task.

As this step is very task-specific, don’t hesitate to reach out to the community for help or guidance.

3. Debug the Task

Reference: srb agent teleop — Teleoperate Agent
Reference: srb agent zero — Zero Agent
Reference: srb agent rand — Random Agent

While developing your task, it is essential to test it to ensure that it behaves as expected. Depending on the action space of your robot, you can either use teleop or zero/rand commands to control the robot. It is also recommended to enable debug visualizations to help you better understand the behavior of the task, and ensure that the environment works with parallel instances:

srb agent teleop -e <your_task_name> env.debug_vis=true env.num_envs=4

srb agent zero -e <your_task_name> env.debug_vis=true env.num_envs=4

srb agent rand -e <your_task_name> env.debug_vis=true env.num_envs=4

4. Use the Task in a Workflow

Reference: Workflows

Now, you are ready to use your task in your desired workflow. Whether you are experimenting with training RL agents or developing a future space mission, your task can be integrated into any workflow that the Space Robotics Bench provides.

Feel free to show & tell us about your task in the community. We are excited to see what you have created!

Adding Terrain Sites + Catalog Entries

This guide walks through extending the curated catalog with new terrain sites, HDRI skydomes, rock sets, and lighting presets. The same shape applies to all four; the manifest at srb/terrain/manifest.yaml is the master record.

Adding a new terrain site

Identify the source DEM. Public-domain or compatibly-licensed only (USGS, NASA, ESA HiRISE-derived). Note the URL, expected sha256, the CRS (e.g., IAU_Moon_2015), and the ground sampling distance in meters.

Edit srb/terrain/manifest.yaml. Add an entry:

sites:
  my_site:
    body: moon  # or mars, earth
    name: "My Site (display name)"
    coords: { lat: 20.5, lon: 30.7 }
    sources:
      dem:
        provider: usgs_astro  # or lroc_pds, pds_geosciences, openplanetary, local
        url: "https://example.org/dem.tif"
        sha256: "<64-char hex>"
        crs: "IAU_Moon_2015"
        gsd_m: 2.0
        license: "Public Domain (NASA/USGS)"
        attribution: "USGS Astrogeology / LROC NAC"
    patches:
      default: { center_xy_m: [256, 256], size_m: 200, lod: medium }
    sky: lunar_default_low      # null if no companion
    lighting: lunar_default      # null to disable
    rocks: lunar_basalt_set      # null to disable
    augment_preset: lunar_default

Validate.
```
srb asset validate-manifest
```
Test the bake.
```
srb asset download terrain my_site
```
Verify the per-site shortcut. srb/assets/scenery/real_terrain_sites.py auto-generates LunarSiteMySite (or MartianSiteMySite) at import time. Smoke-test (run inside the SRB/Isaac Sim Python environment — importing srb.assets.scenery.real_terrain_sites pulls in Isaac Lab; see tests/terrain/unit/test_real_terrain_sites.py for the equivalent gated test):
```
from srb.assets.scenery.real_terrain_sites import LunarSiteMySite
t = LunarSiteMySite()
assert t.asset_cfg.spawn.usd_path
```

Adding a v2 source layer

Real-terrain v2 layers live in a separate, per-body catalog (srb/terrain/catalog/<body>.yaml, e.g. srb/terrain/catalog/moon.yaml) — not the srb/terrain/manifest.yaml sites above. The catalog is reviewed, pinned data: an entry is added only after the product has been downloaded once, sha256’d, and its footprint reviewed by a human (operator action O3). srb/terrain/catalog/refresh.py can propose candidates from upstream indexes (NAC_DTMS_180.SHP / ODE / USGS STAC), but it only ever writes a separate review file — never srb/terrain/catalog/<body>.yaml itself. Promoting a candidate into the real catalog is a manual, by-hand step.

srb/terrain/catalog/moon.yaml carries 19 real, sha256-pinned layers today; mars.yaml and earth.yaml carry 2 each — see Terrain Stacks → Catalog status per body for the exact list. This is curated coverage of a handful of sites, not a global catalog: a resolve against a footprint none of those layers cover still comes back empty until more entries are added (a site request falls back to the v1 single-source path above; an arbitrary-region request degrades to the procedural terrain in envs and errors clearly on the CLI — see Terrain Stacks). LDAM is now curated as imbrium_ldam_50s_1000m. Its detached PDS3 .LBL/.IMG pair is expressed through SourceLayer.sidecars: each file has its own hash and exact basename, acquisition publishes the bundle atomically, and the loader opens the label so GDAL follows ^IMAGE. The real pair was downloaded, hash/size verified, footprint/radiometry reviewed, and production-baked twice. WAC_EMP 643 nm stays priority 3, ahead of priority-4 LDAM 1064 nm where they overlap; that wavelength difference can create a seam. LDAM’s cap:50S adds colour coverage, but no shipped Moon DEM covers the 50S–60S annulus. The NAC_POLE_SOUTH_CM_065/CM_135 mosaics were blocked on the same list for their per-tile size until 2026-08-02, when the one band-1 tile covering Connecting Ridge was downloaded and pinned per epoch (lroc_nac_pole_south_cm065_p892s2250_1m and its cm135 sibling, 8,276,814,528 B each, allow_large_download: true plus reference_only: true — the latter keeps them out of footprint-driven resolution entirely, so they are never fetched as bake input and are reachable only by naming their layer_id, which is how the §6.11 validation study uses them); the remaining tiles stay unpinned, an operator decision per site. A third, SfS A3CLR, was blocked on the same host-allowlist ground until 2026-08-02, when the allowlist was widened and its Connecting Ridge / Haworth DEM + orthomosaic layers were pinned from the Zenodo archive.

Authoring Mars/Earth entries

Mars and Earth catalog entries follow the exact same schema as Moon ones — there is nothing body-specific about SourceLayer itself. The one authoring choice that differs in practice is footprint:

Prefer bbox:<lonmin>,<latmin>,<lonmax>,<latmax> (plain geographic degrees) for non-polar layers — which is most of what Mars and Earth curation will look like (equatorial HiRISE sites, mid-latitude Copernicus/ 3DEP tiles). See Terrain Stacks → Footprints: the bbox: form for the exact parsing/normalization/antimeridian rules. Accepts the 0–360°E convention most planetary product metadata already uses — no manual conversion to ±180° needed.
WKT-in-projected-metres stays for polar canonical CRSs only. A WKT POLYGON(...) footprint’s coordinates must be in the same projected CRS the region resolver uses for that request, which only exists ahead of authoring time for a polar request (the canonical moon_south/ mars_south/earth_south-style CRS, resolved by latitude alone). A non-polar region resolves into a per-request local stereographic CRS generated fresh at request time, which cannot be known when a catalog entry is authored — so a non-polar layer should always use bbox:, never WKT.
cap:<lat><N|S> remains the right shorthand for a full polar-cap product, regardless of body.

Once layers exist, they are consumed two ways: a declared-stack manifest site (below) or an arbitrary region request — RealTerrain(body="moon", lat_deg=..., lon_deg=..., size_m=...) / srb asset download terrain --lat --lon --size-m [--deshade] — which needs no manifest entry at all; the resolver finds coverage by footprint intersection. The region form’s identity, normalization, and fallback rules are documented in Terrain Stacks.

Schema

Validated by srb.terrain.catalog_schema.SourceLayer (srb/terrain/catalog_schema.py):

body: moon
layers:
  - layer_id: ldem_87s # unique within the catalog
    body: moon
    kind: dem # dem | color | mask | quality
    product_family: "LOLA/Kaguya merged polar DEM"
    provider: pgda # ProviderName -- see below
    urls:
      ["https://pgda.gsfc.nasa.gov/data/LOLA_GDR/CYLINDRICAL/IMG/ldem_87s_10m.tif"]
    sha256:
      "ldem_87s_10m.tif": "<64-char hex>" # see "sha256 keys" below
    native_gsd_m: 10.0
    footprint: "cap:83S" # or a WKT POLYGON/MULTIPOLYGON -- see below
    alignment: { source: assumed_zero }
    license: "Public Domain (NASA/GSFC)"
    attribution: "NASA GSFC PGDA"
    size_bytes: 1_400_000_000
    access: cog_remote # cog_remote | download | zip_member
    priority_class: 1 # lower wins -- see below
    allow_large_download: false # curated half of the consent gate

provider accepts the same providers as the v1 manifest above (usgs_astro, lroc_pds, pds_geosciences, openplanetary, local) plus five v2-only additions:

Provider	Hosts	Typical body
`pgda`	`pgda.gsfc.nasa.gov`	moon
`zenodo`	`zenodo.org`	any (zip-member archives)
`uahirise`	`hirise.lpl.arizona.edu`, `www.uahirise.org`, `uahirise.org`	mars
`copernicus_s3`	`copernicus-dem-30m.s3.amazonaws.com`, `copernicus-dem-90m.s3.amazonaws.com`	earth
`usgs_3dep`	`prd-tnm.s3.amazonaws.com`	earth

Every provider is a closed, exact-hostname allowlist — adding a new host means editing that provider module (srb/terrain/sources/*.py), not the catalog. uahirise, copernicus_s3, and usgs_3dep reach public, unauthenticated hosts/S3 buckets only — plain HTTPS GETs, no credential plumbing; a product needing authenticated access cannot be curated with these providers as they stand. Mars USGS products (CTX, HRSC-MOLA) need no new provider — usgs_astro’s existing hosts already cover them.

`priority_class` — lower wins

The resolver sorts candidate layers by the total key (is_synthetic, priority_class, native_gsd_m, layer_id) — never ground-sample distance alone (two products can tie at the same resolution and still need a deliberate order). priority_class defaults to 100; curated entries should set it explicitly. Convention (design spec §4.1/§4.2): DEM rows 1..5, sunlit-color rows 1..4, PSR-color rows 1..4 — lower numbers are preferred, and the lowest fully-covering entry becomes the stack’s role="base". The leading is_synthetic term (Phase 5c, see below) is False for every layer without radiometry: synthetic, so it is inert for every catalog entry described in this doc — priority_class is still the effective tiebreaker for ordinary layers.

`radiometry: synthetic` — the super-resolution slot

radiometry accepts one more value than the photometric-processing classes above: synthetic, reserved for a super-resolution or model-generated DEM/color product — a layer that was not captured from a real sensor. Authoring one means the resolver treats it specially, not that it behaves like an ordinary curated layer:

A footprint-searched request (resolve_stacks, and the mask/quality/PSR-color candidates a declared-stack site still derives by footprint) skips a radiometry: synthetic layer outright unless the caller opts in (RegionRequest.allow_synthetic=True / TerrainSpec.allow_synthetic=True) — it never becomes a resolvable candidate by accident.
When admitted, it still sorts strictly after every non-synthetic candidate, regardless of how low a priority_class you give it — the slot is deliberately a last resort, not a normal priority tier.
A declared-stack site’s dem_stack/color_stack entries are matched by layer_id, not filtered by radiometry — if you name a synthetic layer in a manifest’s declared stack, that is explicit authorial intent and it is never skipped, opt-in flag or not. Only reach for this if the site’s DEM/color truly is a synthetic product and you want every bake of that site to use it.
No shipped catalog entry uses radiometry: synthetic today, and the underlying super-resolution model this slot anticipates does not exist yet in-repo — authoring one is schema-legal but currently only exercised against fixture catalogs in tests.

See Terrain Stacks → Synthetic super-resolution layer slot for the resolver contract and the cache-key effect (allow_synthetic=True appends synthetic=1 to CacheKey.hash12(); False — the default — reproduces every existing golden hash unchanged).

`access` — how a layer’s bytes are fetched

cog_remote — windowed reads over GDAL /vsicurl (srb/terrain/sources/vsicurl.py); the point is that a multi-GB mosaic is never downloaded whole. Use for large products you only need a patch of.
download — a single plain file, fetched whole (BaseHttpProvider.fetch).
zip_member — an archive (currently only exercised via zenodo) fetched whole, then specific pinned members extracted. Use when the product only ships bundled with files you don’t need.

sha256 keys — the D4 convention (read this exactly)

sha256 is a map keyed by basename — defined as urllib.parse.urlsplit(url).path.rsplit("/", 1)[-1] (query string and fragment stripped) — of urls[0], the layer’s primary URL. The rule differs by access, and the schema validator (SourceLayer._validate_sha_map_matches_access) enforces it before a single byte moves:

access != "zip_member" (i.e. cog_remote or download) — the single-file key rule: sha256 MUST have exactly one entry, and that entry’s key MUST equal basename(urls[0]). Zero entries, extra entries, or a mismatched key are all rejected at catalog-load time — one URL, one hash, one key, and the key is not arbitrary.

access == "zip_member" — sha256 MUST contain the archive’s own key (basename(urls[0])) and at least one further key: one per extracted member, keyed by its in-zip path:

access: zip_member
urls: ["https://zenodo.org/records/<id>/files/<archive>.zip"]
sha256:
  "<archive>.zip": "<64 hex>" # the downloaded archive itself
  "dtm/<region>_dem.tif": "<64 hex>" # each extracted member, by in-zip path

Getting this wrong is not a soft warning: the fetch path’s CatalogLayerPlan.archive_sha256 does a bare sha256[basename(urls[0])] lookup, so a conforming-but-unpinned layer would KeyError at fetch time if the schema validator hadn’t already caught it at load time. srb asset stack refuses to build a plan from a catalog file that fails this validation at all.

Compute a real hash the same way as v1:

curl -L "<url>" | sha256sum

The placeholder "f" * 64 is a hard refusal here too, exactly like the v1 manifest (PLACEHOLDER_SHA256, srb/terrain/sources/acquisition.py) — a catalog layer with any placeholder sha256 value (archive OR member key) is refused before a byte moves.

`allow_large_download` + `SRB_TERRAIN_LARGE_OK`

Unlike the v1 manifest — where allow_large_download: true alone is sufficient, and that behavior is unchanged (D6) — a v2 catalog layer needs both halves to agree before an unapproved-size fetch proceeds:

allow_large_download: true on the layer itself (curated — set this when you add the entry, asserting you’ve reviewed the size), and
the operator’s SRB_TERRAIN_LARGE_OK=1 environment variable at fetch time (never baked into the catalog — a per-run opt-in).

Without both, a layer whose estimated fetch exceeds 1 GiB (MAX_UNAPPROVED_SOURCE_BYTES) is refused (LargeDownloadConsentRequiredError) rather than silently capped or prompted — nothing in this codebase prompts interactively (D11: SRB runs headless in containers and CI, where a prompt is an unbounded hang, not a question).

Do not confuse this real, AND-based enforcement gate (check_layer_consent / layer_max_bytes, srb/terrain/sources/acquisition.py) with srb asset stack’s printed “consent required” line, which is a descriptive, OR-based preview (--allow-large OR SRB_TERRAIN_LARGE_OK) that never touches the fetch path — see Real Terrain Assets.

`mask_encoding` — required in practice for every `kind: mask` layer

Read this before adding any kind: mask entry. A mask layer’s raw pixel values are product-specific DNs. Without a mask_encoding block the raster is passed through to fusion raw, byte-for-byte — the schema’s default is None and absence is defined as “unchanged” — and srb.terrain.fusion.color.psr_mask’s predicate is simply finite and nonzero. For a product whose “not masked” DN is a large nonzero number, that predicate is true nearly everywhere, and the mask silently covers the whole region instead of the small part it describes. This is not hypothetical: it is exactly how the entire lunar south pole was classed as permanently shadowed until the field was added.

The worked example is pgda_lpsr_85s_60m in srb/terrain/catalog/moon.yaml. The raw int16 LPSR raster the URL actually serves is tri-valued — 20000 means permanently shadowed, -32768 means nodata, and everything else (in practice -20000) means not shadowed:

  kind: mask
  # ...
  mask_encoding:
    masked_values: [20000]     # -> 1.0  ("inside the mask")
    nodata_values: [-32768]    # -> NaN  ("no observation")
                               # everything else -> 0.0 ("outside the mask")

With the block, -20000 decodes to 0.0 and drops out; without it, -20000 is finite and nonzero, so 99.99998% of the raster reads as masked. The decode happens at ingest (srb.terrain.ingest.mask.decode_mask_dn), before any array reaches fusion/.

Three rules when authoring one:

Values are compared for exact equality against the raster’s own pixel values read as float, so list the product’s real DNs — not a range, not a threshold. Non-finite and duplicate values are rejected at load.
A typo’d key is a hard error, not a silent fallback. SourceLayer is extra="forbid", so mask_encodding: fails catalog validation rather than loading with mask_encoding is None and quietly restoring the bug.
Check the bytes the URL actually serves, not the product’s documented canonical form. PGDA’s LPSR is documented as a {0, 1, 255} uint8 mask; the .TIF at the pinned URL is the raw int16 one. Open the file and look at its histogram before writing the block. Keep the constants in sync with srb/core/ephemeris/importers.py::_read_pgda_lpsr, which decodes the same product independently.

`footprint` — three forms, and the CRS requirement (D12)

cap:<lat><N|S>, e.g. cap:83S — a spherical cap poleward of that latitude. The common case for polar mosaics.
A WKT POLYGON(...) / MULTIPOLYGON(...) string, with coordinates in the same projected CRS (meters) the region resolver uses — never latitude/longitude degrees. Only practical for polar layers (below).
bbox:<lonmin>,<latmin>,<lonmax>,<latmax> — plain geographic degrees, body-agnostic (Phase 5b, D12 lift). Recommended for non-polar layers — see Authoring Mars/Earth entries above and Terrain Stacks → Footprints: the bbox: form for the full parsing/normalization/antimeridian rules.

A cap:/WKT footprint and the region it is tested against must share one projected CRS — comparing a footprint’s projected meters against a request expressed in degrees would silently produce a nonsense intersection; bbox: sidesteps this entirely by staying in degrees and letting the resolver project each sample point back before comparing. Every request now projects into a body-aware CRS before the footprint math runs (Phase 5b, D12 lift): a polar request (|lat| >= 60) uses the canonical moon_south/moon_north- style CRS (IAU_2015:30135/:30130, and the Mars/Earth equivalents — see Terrain Stacks → Bodies and canonical CRSs), and any other request uses a per-request local stereographic CRS centered on its own lat/lon — there is no longer a structural non-polar gate, only a genuine no-covering-footprint miss, which still returns an empty plan with a warning and falls back to the v1 single-source path.

Validate

srb asset stack --body moon --lat -89.46 --lon 137.3 --size 2000
srb asset stack --body moon --lat -89.46 --lon 137.3 --size 2000 --json

A malformed entry (bad sha256 shape, unrecognized footprint, duplicate layer_id, unknown provider) raises a pydantic.ValidationError at load time, before any plan is built.

Adding a v2 site with a declared stack

A v2 site is different from a v2 source layer (above): the layer is reviewed, pinned catalog data (srb/terrain/catalog/<body>.yaml); the site is a srb/terrain/manifest.yaml entry that names which catalog layers to fuse for one bake, the same way a v1 site names one dem/ortho. This section assumes the layers you want to reference already exist in the catalog (curated, per the section above, or — for local testing — a hand- built catalog like the one tests/terrain/fixtures/fusion_stack.py builds).

Bump the manifest to version: 2. This is required before ANY site may declare a stack, and is backward compatible: existing v1 sites in the same file are completely unaffected (Sources still validates “exactly one of dem / dem_stack” per site, independent of the manifest’s own version).
```
version: 2
sites:
  apollo17: { ... }  # existing v1 sites, unchanged
```
Add the site, declaring dem_stack (exactly one role: base, plus zero or more role: inset) and, optionally, color_stack (same shape), instead of the v1 sources.dem/sources.ortho:
```
sites:
  connecting_ridge:
    body: moon
    name: "Connecting Ridge (v2 fused)"
    coords: { lat: -89.46, lon: 137.3 }
    sources:
      dem_stack:
        - { layer_id: ldem_87s, role: base }
        - { layer_id: nac_dtm_esall_cr1, role: inset }
      color_stack:
        - { layer_id: cm_avg_87s, role: base }
        - { layer_id: shadowcam_cmosaic_cr1, role: inset }
    patches:
      default: { center_xy_m: [0, 10309.97], size_m: 2000, lod: medium }
      # -- or, equivalently, a lat/lon patch center instead of projected xy:
      # default: { center_latlon: [-89.66, 0.0], size_m: 2000, lod: medium }
    sky: lunar_default_low
    lighting: lunar_default
    rocks: lunar_basalt_set
    deshade: true # optional: bake illumination-corrected colour by default
```
The optional deshade: true makes every bake of this site divide each sunlit colour layer by a modelled hillshade of the fused DEM at that layer’s catalogued sun_azimuth_deg/sun_elevation_deg (PSR layers are never deshaded; a layer without catalogued angles is skipped with a warning). Precedence with the request-level knob is manifest or request — a RealTerrain(..., deshade=True) request can turn deshading on for a site that doesn’t declare it, but can never turn a manifest-declared deshade: true off. The effective flag is part of the cache key. See Terrain Stacks → Deshade.

Every layer_id must exist in that body’s catalog — an unknown one raises at resolve time, naming the missing layer. role (not list order) decides the base; StackRef.max_gsd_m can drop an overly coarse candidate layer with a warning if you need to bound resolution.
Validate and bake exactly like a v1 site — same commands, same srb asset validate-manifest / srb asset download terrain <site> / auto-generated LunarSiteConnectingRidge shortcut class. Nothing about the CLI or the RealTerrain constructor differs; bake_or_load detects the declared stack (sources.dem is None) and routes to the fusion engine automatically.
What comes out differs from a v1 bake: meta.json carries a fusion block, provenance.tif sits alongside terrain.usd, and ATTRIBUTION.txt lists one block per contributing layer instead of a single string. See Real Terrain Assets → the fusion engine for what each of those actually contains, the mesh/texture budget tiers, and the QC/degradation rules (a demoted or unreachable inset drops with a warning and the bake still succeeds; an unreachable base has no v1 fallback to take instead, and raises).

This only works once the layers you name actually exist in a real catalog. srb/terrain/catalog/moon.yaml now carries 19 real layers (operator action O3 curated them — see Adding a v2 source layer above), but the connecting_ridge example’s own layer ids (ldem_87s, nac_dtm_esall_cr1, cm_avg_87s, shadowcam_cmosaic_cr1) are still illustrative — they don’t match any real, shipped layer id, and no dem_stack/color_stack manifest site has actually been declared against the real catalog yet (manifest.yaml is still version: 1). What is proven against the real catalog today is the arbitrary-region request form (no manifest site) — srb asset download terrain --lat -89.66 --lon 0.0 --size-m 2000 --body moon completes a real fused bake end to end; see Terrain Stacks → Catalog status per body.

Adding a new HDRI skydome

srb/terrain/hdri/catalog.yaml:

skydomes:
  my_dome:
    body: moon
    url: "https://example.org/dome.hdr"
    sha256: "<64-char hex>"
    format: hdr        # or png, exr
    intensity: 1.0
    resize_to: 2048
    license: "Public Domain"
    attribution: "Source name"

EXR support requires the optional imageio[freeimage] dependency (pip install srb[terrain-extras]); PNG and HDR work out of the box.

Adding a new rock set

srb/terrain/rocks/catalog.yaml:

sets:
  my_set:
    body: moon
    rocks:
      rock_a:
        url: "https://example.org/rock_a.glb"
        sha256: "<64-char hex>"
        format: glb       # or obj, ply, stl
        target_face_count: 5000
        license: "Public Domain"
        attribution: "Source"
      rock_b:
        url: "..."
        ...

The pipeline decimates each rock to target_face_count (default 5000), welds duplicate vertices, recenters at centroid, rescales the AABB diagonal to 1.0. Caller scales at scatter time.

Adding a new lighting preset

srb/terrain/lights/catalog.yaml (no fetch — pure-config):

presets:
  my_preset:
    body: moon
    sun_elevation_deg: 30
    sun_azimuth_deg: 90
    sun_intensity_lux: 130000
    sun_color_temperature_k: 5800
    sun_angular_diameter_deg: 0.53
    earthshine_intensity_lux: 200      # optional
    earthshine_color_temperature_k: 7500
    license: "Public Domain"
    attribution: "Synthesized from ephemeris X"

Sun pose can be derived from a real ephemeris for a given mission timestamp

site coordinates, or synthesized for an artistic look.

sha256 backfill workflow

For new entries the placeholder convention is "f"*64. To compute the real hash:

curl -L "<url>" | sha256sum

Update the manifest, re-run srb asset validate-manifest, and confirm srb asset download terrain <site> succeeds.

Common mistakes

Wrong CRS string. Use the IAU 2015 names: IAU_Moon_2015, IAU_Mars_2015. Misspellings raise MissingCRSError or DemLoadError during ingest.
Patch size larger than DEM. Cropping outside the DEM extent raises EmptyHeightmapError. Pick smaller patch_size_m or different center_xy_m.
Forgetting attribution on user-supplied DEMs. License compliance is a hard refusal — bakes won’t run.
Stale companion catalog references. sky:/lighting:/rocks: must match a name in the corresponding catalog. Use null to disable.

Attributions

Asset	Source	License	Author
`canadarm3`	SRB	–	SRB Contributors
`franka`	SRB	–	SRB Contributors
`kinova_gen3n7`	Isaac Lab	NVIDIA	NVIDIA
`so_arm100_d5`	SRB	–	SRB Contributors
`so_arm100_d7`	SRB	–	SRB Contributors
`unitree_z1`	SRB	–	SRB Contributors
`ur10`	Isaac Lab	NVIDIA	NVIDIA
`ur3`	Isaac Lab	NVIDIA	NVIDIA
`ur5`	Isaac Lab	NVIDIA	NVIDIA
`vispa`	SRB	–	SRB Contributors

Contributors

Andrej Orsula

Citation

If you use this project in your research, please cite the following publication:

@article{orsula2025space,
  title   = {{Space Robotics Bench: Robot Learning Beyond Earth}},
  author  = {Orsula, Andrej and Geist, Matthieu and Olivares-Mendez, Miguel and Martinez, Carol},
  year    = {2025},
  journal = {arXiv:2509.23328},
  doi     = {10.48550/arXiv.2509.23328},
}

Community

We encourage you to join our Discord community to discuss the Space Robotics Bench, share your projects, ask questions, and collaborate with other researchers and developers. We are excited to see what you create with SRB!

Invite Link: https://discord.gg/p9gZAPWa65

Guidelines

Please adhere to the following guidelines when participating in the community:

Be respectful and considerate to all members
Keep your discussions relevant

We look forward to interacting with you in the community!

Troubleshooting

Runtime Errors

Driver Incompatibility

If you encounter one of the following error messages:

[Error] [carb.graphics-vulkan.plugin] VkResult: ERROR_INCOMPATIBLE_DRIVER

XlaRuntimeError: UNIMPLEMENTED: /usr/local/lib/python3.11/site-packages/jax_plugins/xla_cuda12/../../nvidia/cuda_nvcc/bin/ptxas ptxas too old. Falling back to the driver to compile.

This indicates that your NVIDIA driver is incompatible with Omniverse or the learning frameworks. To resolve the issue, update your NVIDIA driver according to the Isaac Sim driver requirements and the specific requirements of the deep learning frameworks you are using.

GLXBadFBConfig Error

If your srb agent simulated workflow crashes and you encounter the following error message, this might indicate that your system’s OpenGL version is not compatible with the requirements.

X Error of failed request:  GLXBadFBConfig
  Major opcode of failed request:  150 (GLX)
  Minor opcode of failed request:  0 ()
  Serial number of failed request:  133
  Current serial number in output stream:  133
There was an error running python

To resolve this, you can set the MESA_GL_VERSION_OVERRIDE environment variable when running the srb agent command as shown below:

MESA_GL_VERSION_OVERRIDE=4.6 srb agent ...

Alternatively for users of the provided Docker setup, you can set the environment variable for the entire Docker container by running:

./.docker/run.bash -e MESA_GL_VERSION_OVERRIDE=4.6

Slow Startup

The first launch of Isaac Sim can take several minutes due to shader compilation and asset caching. Subsequent launches are significantly faster. If startup remains slow, see this issue for potential workarounds.

If a simulated workflow selects real terrain and the cache is cold, SRB may need to fetch and bake the selected terrain before launch. Disable terrain auto-fetch for interactive runs when you want SRB to warn once and fall back to the procedural surface instead of contacting remote terrain providers:

SRB_TERRAIN_AUTO_FETCH=0 srb agent teleop -e _ground

With the provided Docker development wrapper and a virtual display:

DISPLAY=:99 DOCKER_TTY=false .docker/dev.bash -e SRB_TERRAIN_AUTO_FETCH=0 srb agent teleop -e _ground

The Docker development wrapper defaults to LOG_LEVEL=info so expected Isaac and SimForge debug logs do not hide actionable warnings. For a one-off verbose run, prefix the command with LOG_LEVEL=debug.

Hidden Isaac Startup Logs

SRB suppresses several known noisy Isaac Sim and PhysX startup log sources during srb agent workflows so important warnings remain visible. If you are diagnosing an upstream Isaac or Kit startup issue and need the raw vendor logs, disable these filters for that run:

SRB_DISABLE_ISAAC_LOG_FILTERS=1 srb agent teleop -e _ground

With the provided Docker development wrapper and a virtual display, pass the same variable into the command environment:

DISPLAY=:99 DOCKER_TTY=false .docker/dev.bash -e SRB_DISABLE_ISAAC_LOG_FILTERS=1 srb agent teleop -e _ground

Unexpected Behavior

Teleoperation Stuck

During teleoperation with the keyboard, if you change your window focus, Omniverse may fail to register a button release, causing the robot to move continuously in one direction. To fix this, press the L key to reset the environment.

Haven’t found a solution to your problem? You can search for help or ask any questions by joining our Discord community or by seeking assistance through GitHub Issues.

Keyboard shortcuts

Space Robotics Bench