Sim-to-Real Transfer

The Space Robotics Bench provides a streamlined workflow for deploying agents trained in simulation directly onto physical hardware. This is managed through the real_agent command-line interface, which runs a hardware-interfacing equivalent of a simulated environment. This allows various workflows, including the deployment of a trained RL policy, to be executed on a real robot with minimal changes.

1. Train your Agent in Simulation

Reference: Reinforcement Learning Workflow

The first step is to train a policy in simulation. The goal is to produce a stable policy checkpoint. All RL frameworks and algorithms integrated into SRB are supported by this sim-to-real workflow.

Let’s train a Dreamer agent for the waypoint_navigation task with the Leo Rover on 512 parallel environments. Since we are deploying in an on-Earth facility, we will also specify the earth gravity setting. (For the best results, it is highly recommended to use a combination of Domain Randomization and Procedural Generation over the course of the training. This creates a more robust agent that is better prepared for the complexities of the real world.)

srb agent train --headless --algo dreamer --env waypoint_navigation env.robot=leo_rover env.num_envs=512 env.domain=earth

2. Generate Sim-to-Real Bridge

Reference: srb real_agent gen — Generate Sim-to-Real Bridge

This key step creates the bridge between the simulation and the real world. The srb real_agent gen command inspects a simulated Gymnasium environment and automatically writes a lightweight, real-world counterpart that does not depend on the simulation backend.

You can specify which default HardwareInterface modules your robot uses via the --hardware flag. These are the drivers that communicate with your robot’s software, e.g., via ROS 2. For the Leo Rover, we will need an interface to send velocity commands (ros_cmd_vel) and one to receive pose information (ros_tf). Furthermore, we will use the ros_mw interface to expose ROS 2 service calls for pausing and resuming the agent. It is important to specify the robot here so that its parameters, such as action scaling, can be correctly extracted for the real-world environment.

Tip: To see every available hardware interface ID, run srb real_agent gen --help, or browse the sim_to_real/hardware directory.

srb real_agent gen --env waypoint_navigation env.robot=leo_rover --hardware ros_cmd_vel ros_tf ros_mw

This command launches a temporary headless SRB session. It loads the environment, inspects its APIs, and then writes a new Python file inside the sim_to_real/env directory. This Python file defines a RealEnv class and registers it in Gymnasium under the srb_real/ namespace.

3. Deploy and Evaluate on Hardware

With the bridge generated, you can deploy the agent to your robot. The real_agent command does not launch a simulation. Instead, it runs the generated RealEnv, which connects directly to your hardware.

To evaluate the policy trained in the first step, run the following command.

srb real_agent eval --env waypoint_navigation --algo dreamer

The RealEnv will instantiate the ros_cmd_vel, ros_tf, and ros_mw interfaces. When the policy produces an action, the environment routes it to the RosCmdVelInterface, which publishes it as a ROS 2 message. It then gets the latest pose from the RosTfInterface to use as an observation for the policy’s next step.

4. Advanced Workflows and Use Cases

The real_agent tool is not just for evaluation. It enables several powerful workflows for research and development.

Debugging with Zero and Random Agents

Before deploying a fully autonomous policy, you can use the zero and rand agents to quickly test your hardware setup. The zero agent does nothing, while the rand agent sends random actions to the robot.

srb real_agent zero --env waypoint_navigation

srb real_agent rand --env waypoint_navigation

Fine-Tuning on Real Data

Note: While SRB supports fine-tuning on real data via the srb real_agent train --continue command, this workflow is still under active development and should be considered experimental.

5. Additional Task Examples

The same workflow generalizes to mobile-manipulation and locomotion tasks. The following subsections show representative deployment commands for the excavation and terrain landscaping tasks, along with notes on expected robot setups, hardware interfaces, and evaluation metrics to monitor.

5.1 Excavation

The excavation task is designed for a Husky + Kinova mobile manipulator equipped with a scoop end-effector payload. Two variants are supported, depending on the motion-generation stack used on the real robot.

OSC (operational-space control) variant:

srb real_agent eval --env excavation --algo skrl_ppo \
  --hardware ros_depth_heightmap ros_tf ros_imu ros_kortex_cartesian ros_mw

skrl_ppo_rnn remains a reserved, fail-closed label because SRB has no recurrent model, sequence-memory, or hidden-state reset contract for the pinned SKRL runtime. The command above illustrates adapter wiring only; it is not hardware-readiness evidence.

The hardware interfaces used here:

• ros_depth_heightmap — Subscribes to a RealSense depth stream and projects it into a local heightmap grid.
• ros_tf — Receives the base pose from an external mocap/OptiTrack source.
• ros_imu — Consumes IMU feedback for base orientation and angular rates.
• ros_kortex_cartesian — Drives the Kinova arm via Cartesian admittance and exposes FK and contact-force readings.
• ros_mw — Provides middleware service calls for pausing and resuming the agent.

Joint-velocity variant:

srb real_agent eval --env excavation --algo skrl_ppo_rnn \
  --hardware ros_depth_heightmap ros_tf ros_imu ros_kortex_joint_vel ros_mw

Additional interfaces relative to the OSC variant:

• ros_kortex_joint_vel — Sends joint-velocity commands to the Kinova arm. Use this variant when an external motion-generation layer (e.g. NVIDIA fabrics) is responsible for translating policy outputs into reactive joint targets.

Evaluation metrics the operator should watch during rollouts: success_rate, terminal_reward, particle_count_in_scoop_volume, and action_smoothness.

5.2 Terrain Landscaping

The canonical landscaping deployment is RaphRover + prismatic-velocity RaphShovel with one external heightmap source. It has no onboard depth camera, no IMU observation, no blade-pitch channel (measured or synthetic), no previous-action feedback, and no Leo Rover profile. The legacy Leo adapter stack is kept only under the separately named hyperparams/validation/terrain_landscaping_leo.yaml profile; it does not share the canonical task IDs and is not generated by the commands below.

There are two canonical task IDs, and the generated modules are distinct:

The general task must never be relabelled with the crater hash. It now owns a hashed per-episode mission manifest of its own, so RealEnv construction still demands one at deployment time (REQUIRES_RUNTIME_TARGET_MANIFEST) rather than borrowing the crater’s.

# Regenerate the checked-in bridge modules (writes the module in place)
srb real_agent gen --env terrain_landscaping_crater
srb real_agent gen --env terrain_landscaping

# Verify the checked-in modules are byte-identical to a fresh render
srb real_agent gen --env terrain_landscaping_crater --check
srb real_agent gen --env terrain_landscaping --check

Both commands run unqualified. The general task previously needed a particle-height override, because plain gen --env terrain_landscaping aborted during env.reset() with ExternalHeightmapFrameError: its particle bed spilled off an undersized containment surface and heaped past the D5 elevation envelope. That defect is fixed in the task itself, so no override belongs in these commands and none must be reintroduced.

--check is read-only: it renders and formats a candidate, byte-compares it with the checked-in module, prints a unified diff, and exits non-zero on drift without touching the file. It needs the repo’s pinned formatter (ruff) on PATH — without one it refuses to compare rather than report drift that an unformatted candidate would manufacture. srb real_agent gen --env ALL --check runs that gate across every cached environment and exits non-zero if any has drifted, or if the environment cache is empty (a gate that inspected nothing is not a pass). Generated modules whose task is no longer registered are absent from the cache and therefore outside the batch’s reach.

Generated schema. Generation derives the module from the frozen IO contract through the typed env.srb_spec seam — never from one sampled step return — and refuses to emit np.finfo dtype-extrema bounds:

• 3 actions, all normalized velocities: robot/cmd_vel on [0:2] (linear, angular) and payload/joint_vel on [2:3].
• 7 actor observation leaves with finite physical bounds: proprio_dyn/heightmap_current_global, proprio_dyn/heightmap_target_global, proprio_dyn/heightmap_current_local, proprio_dyn/heightmap_target_local, proprio/mission_pose, proprio/base_velocity, and proprio_dyn/heightmap_age_s — 1287 float32 in contract order.
• The assembled SINGLE_OBSERVATION_SPACE is four map outputs plus one packed vector; that is an assembly of the seven leaves, not seven top-level outputs.
• Class constants ENV_ID, IO_SCHEMA_FINGERPRINT, TARGET_MANIFEST_SHA256, REQUIRES_RUNTIME_TARGET_MANIFEST, and DEPLOYMENT_REQUIREMENTS record the deployment identity. Keep target_map_sha256, reset_layout_sha256, manifest_sha256, and io_schema_fingerprint distinct — they are four different hashes and conflating them hides real drift.

Deployment is gated, and nothing in this repository opens the gate. Constructing the canonical landscaping RealEnv validates the union of the capability tags claimed by the supplied hardware interfaces and capability providers before it acquires a ROS node, starts any hardware, or lets a caller load a policy. All four tags are required:

• raph.drive_velocity
• raph.shovel_prismatic_velocity
• landscaping.external_heightmap_batch
• landscaping.observable_task_evaluator

Exactly one shipped component claims a tag: srb.interfaces.sim_to_real.validation.landscaping.LandscapingRealEvaluator claims landscaping.observable_task_evaluator. Pass it as RealEnv(..., task_evaluator=...); it grades the accepted external frames against the verified target manifest, owns the reward, and refuses a policy whose model artifact declares a different task id, IO-schema fingerprint, target manifest, reset layout, or normalization policy — before ROS acquisition, hardware start, or inference.

The three remaining tags have no claimant, so construction still raises a typed DeploymentNotReadyError listing the exact missing tags. That is the intended state: the Raph ROS drive/shovel commands and the live external mapper are operator dependencies that must be recorded from the lab first — command topic and message type, unit and sign convention, saturation behaviour, watchdog/timeout semantics, acknowledgement mechanism, and measured joint-state feedback evidence.

Note: Simulation and fake-adapter tests do not prove Raph physical-hardware readiness. Nothing below the gate above has been validated against real hardware.

Real Raph validation is unavailable, and stays unavailable until the external dependencies land. No hardware evidence of any kind exists for any landscaping task. The hardware-free half is complete — the observable task evaluator, the five session-abort outcomes, the horizon-truncation semantics, the model-artifact refusal, and a crater validation spec that resolves its real environment and fails closed. The live half was never started: no live Raph drive/shovel or mapper contract has been recorded, and no lab run exists. Passing every simulation gate on this page would still not imply hardware readiness.

The canonical validation spec is hyperparams/validation/terrain_landscaping_crater.yaml. It declares no hardware, so running it fails closed on the capability preflight (exit 4) until the live adapters exist. Its observable metrics are success_rate, final_grading_mae_m, final_fraction_within_tolerance, action_smoothness, and map_age_s; the final step’s info additionally carries the diagnostic central-ROI and local MAE, sustained success, the four distinct hashes, and the calibration version.

For the full readiness picture — which learners are usable, what the observation contract actually is, and which gates are blocked — see Terrain Landscaping Training Readiness. In particular: TD-MPC2 is disabled for both landscaping task IDs and fails closed before any expensive side effect, and no learned landscaping policy is claimed, so a real crater session driven by a current policy would be expected to report a final grading MAE close to the zero-action baseline and success_rate = 0. The harness would report that as exit 1, correctly; A8a proves the reporting is truthful, not that anything can pass.

6. Validation Workflow

Advanced / reference. Sections 1–3 above are all you need to deploy and evaluate a policy. This section documents the optional structured-validation harness (release gating, drift tracking, dashboards) and is reference-level detail — skip it unless you are setting up repeatable release validation.

Once a checkpoint is deployed via srb real_agent eval, you can run a structured validation session that produces a machine-readable pass/fail, multi-modal telemetry (rerun.io, JSONL, optional W&B), and a static dashboard tracking drift across releases.

6.1 Capture a sim baseline

The sim baseline is captured once per checkpoint by reusing Isaac Lab’s existing RerunVisualizer:

srb agent eval --env excavation --algo skrl_ppo_rnn --model <ckpt> \
    env.num_envs=1

The checkpoint is passed with --model (not --checkpoint), and the parallel env count is a Hydra override (env.num_envs=). The episode count, .rrd baseline recording (rerun.record_to_rrd), and W&B logging are configured through the validation spec YAML (see §6.2) rather than as srb agent eval flags. The W&B summary scalars (success_rate, terminal_reward, action_smoothness, etc.) are referenced later by the validation harness via the run id; the .rrd file enables twin-replay debugging.

6.2 Validation spec YAML

The validation spec is the persistent Interface between an operator, a generated RealEnv, and the validation harness. CLI flags only override a few run-time fields; the YAML keeps the task, policy, hardware adapters, criteria, telemetry, baseline, and drift contract together.

task: excavation
algo: skrl_ppo_rnn
checkpoint: REQUIRED_AT_RUNTIME
hardware:
  - ros_depth_heightmap
  - ros_tf
  - ros_imu
  - ros_kortex_cartesian
  - ros_mw
metrics_to_track:
  - success_rate
  - terminal_reward
  - action_smoothness
thresholds:
  success_rate_min: 0.5
  sim_real_ratio_min: 0.7
  action_smoothness_max_ratio: 1.5
  crash_rate_max: 0.0
  drift_z_max: 3.0
sim_baseline:
  wandb_run_id: entity/project/run-id
  local_rrd: baselines/excavation/<sha>.rrd
rerun:
  app_id: srb-validation
  web_port: 9090
n_episodes: 10
drift_window: 5
storage_root: logs/real_validation
notes: excavation release gate

Top-level fields:

Threshold fields:

Rerun fields:

Simulation baseline fields:

Fault-injection fields:

6.3 Run validation on the real robot

srb real_agent validate \
    --spec hyperparams/validation/excavation.yaml \
    --checkpoint <real-or-shared-ckpt> \
    --episodes 10 \
    --storage-root /tmp/srb_validation

Validation accepts --spec, --checkpoint, --episodes, --wandb, --no-tensorboard, --no-rerun-web, --storage-root, --post-status, and --dry-run. Local artifacts and TensorBoard remain the default; W&B requires explicit opt-in.

--dry-run is explicitly non-hardware and never passes. It skips environment and policy instantiation, then finalizes with pass_overall: false, hardware_evidence: false, exit_code: 4, and metrics.n_episodes: 0. Its three criteria carry status skipped, and the badge is red with a non-hardware evidence message. The session is not recorded in the drift store. Treat it as a wiring smoke artifact only; it is not real-data or hardware evidence and cannot be used as a validation pass.

The harness:

1. Spawns a rerun web viewer at http://localhost:9090 for live in-session inspection.
2. Runs <--episodes> rollouts on the real robot via the RealEnv generated earlier with srb real_agent gen.
3. Writes the validation session and storage-root artifacts described in the next section.
4. Compares against the sim baseline using three criteria:
   • (a) Threshold: success rate ≥ minimum, sim/real ratios in bounds. The success-rate floor is evaluated even when no sim baseline is configured — only the sim/real ratio half is skipped — so a session without a single successful episode never exits 0.
   • (b) Crash: no policy_nan, hardware_event, disk_full terminations, and no safety-relevant session abort (sensor_stale, sensor_incomplete, calibration_mismatch, manifest_mismatch)
   • (c) Drift: z-score of all tracked metrics ≤ thresholds.drift_z_max over the last K sessions

Episodes that end at the environment’s configured horizon are recorded as time_limit_truncation (a truncation carrying the final observable metrics), never as the harness’s wall-clock timeout. Out-of-MDP session aborts get their own term reasons and per-reason counts in the session metrics: sensor_stale, sensor_incomplete, calibration_mismatch, manifest_mismatch, and operator_abort. None of them is a task termination or a truncation; an operator stop fails the success-rate floor rather than the crash criterion.

1. Exits with the validation result code:

A validation session always runs the generated srb_real/<task> environment; the command registers those ids itself before constructing one. If the real environment cannot be resolved, validation raises — it never substitutes the srb/<task> Isaac simulation task, because the session would still be written up as a real-validation report.

Which shipped specs can actually run. Removing that simulation fallback was correct — it was writing simulation results up as hardware proof — but it also means most shipped specs now stop at pre-flight. Only a spec whose task has a generated module under srb/interfaces/sim_to_real/env/ is runnable:

The pre-flight error message suggests srb real_agent gen. That remedy is correct only for the rows above where a deployment profile exists (srb/interfaces/sim_to_real/deploy_profiles/generation.py); for the three spacewalk specs there is no profile at all, so nothing can be generated for them today. Each non-runnable spec carries a # NOT RUNNABLE header saying so. No shipped spec is claimed to have been validated on hardware.

6.4 Validation artifacts

Validation writes one session directory plus two storage-root artifacts:

logs/real_validation/
+-- _index.jsonl
+-- index.html
+-- <task>/
    +-- <uuid>_<utc-timestamp>/
        +-- current.json
        +-- generations/
            +-- <generation-uuid>/
                +-- spec.yaml
                +-- episodes.jsonl
                +-- rollout.rrd
                +-- tensorboard/
                +-- summary.json
                +-- report.md
                +-- badge.json

Session artifacts:

Storage-root artifacts:

6.5 Cross-session dashboard

A static HTML dashboard at logs/real_validation/index.html is regenerated on every run. Open it in a browser to see all sessions, their pass/fail state, and trend charts per (task, algo, metric).

6.6 Twin replay (debugging)

When a session fails, compare it against the sim baseline:

srb real_agent replay-twin \
    --session logs/real_validation/excavation/<uuid>_<ts>/generations/<generation-uuid> \
    --baseline baselines/excavation/<sha>.rrd

The rerun viewer opens with sim ghost (transparent) overlaid on real (solid), time-aligned by episode index.

6.7 Recovering a partial session

If a session crashes mid-rollout (disk full, network loss, etc.), summary.json will be missing. Rebuild it from spec.yaml and episodes.jsonl:

srb real_agent rebuild-summary logs/real_validation/excavation/<uuid>_<ts>/generations/<generation-uuid>

6.8 Safety reminder

The validation harness observes safety events (estop, joint-limit violations, contact-force spikes) but does not enforce them. Physical safety remains the operator’s and hardware bring-up’s responsibility — the harness records these events and counts them toward criterion (b), but does not replace deadman switches, joint-limit watchdogs, or estop circuitry.

7. Creating Custom Hardware Interfaces

You can easily support custom sensors or actuators. To create a new interface, add a new Python file in the sim_to_real/hardware directory. Your new class should inherit from the HardwareInterface base class.

You will need to implement a few key methods:

• start to initialize your hardware connection.
• apply_action to send commands.
• observation to get sensor data.
• close to clean up connections.

The system will discover your new interface automatically, making it available to the --hardware flag.