Sim-to-Real Transfer

Hardware acceptance requires native transport and operator-reviewed physical-stop acceptance (ADR-0020). Examples and host tests do not establish readiness.

RealEnv checks adapter health before dispatch and after observation; degraded or nonhealthy fallback frames fail the live check. Prepared drive, blade, and direct Kortex adapters validate before sending and commit filter state only after successful sends. Partial sends across transports latch the session and trigger stop attempts; legacy adapters lack this guarantee. Kortex reports submitted values, not measured motion. Zero Cartesian scales are honored.

1. Train your Agent in Simulation

Reference: Reinforcement Learning Workflow

Train a policy in simulation. Real-agent evaluation is subject to algorithm, backend, and preflight gates; skrl_amp is gated. Dreamer train_real fine-tuning is experimental. Example: Leo Rover, 512 environments, Earth gravity.

srb agent train --headless --algo dreamer --env waypoint_navigation env.robot=leo_rover env.num_envs=512 env.domain=earth

2. Generate Sim-to-Real Bridge

Reference: srb real_agent gen — Generate Sim-to-Real Bridge

srb real_agent gen inspects a simulated Gymnasium environment and writes a hardware counterpart without a simulation-backend dependency.

Select drivers with --hardware (for example, Leo Rover ros_cmd_vel, ros_tf, and ros_mw). Specify the robot so generation extracts parameters such as action scaling.

Generation writes the module in place. It launches a temporary headless SRB session and overwrites the module in sim_to_real/env. Verify the profile against the target robot (ADR-0001).

Tip: To see every available hardware interface ID, run srb real_agent gen --help, or browse the sim_to_real/hardware directory.

srb real_agent gen --env waypoint_navigation env.robot=leo_rover --hardware ros_cmd_vel ros_tf ros_mw

The generated module defines RealEnv, defaults hardware IDs, and registers under srb_real/. RealEnv checks adapter frame pairs before startup.

Registration is deferred: imports register no srb_real/ id. srb real_agent calls srb.interfaces.sim_to_real.env.register_all() to sweep modules. Thus srb_real/ environments do not appear in srb ls env or environment pages, which list only srb/.

3. Deploy and Evaluate on Hardware

After verifying bridge routes and profile, real_agent runs generated RealEnv with hardware interfaces; it does not launch simulation.

srb real_agent eval --env waypoint_navigation --algo dreamer

On exit or signal, runtime closes the environment before releasing ROS, latches the session, and submits final stops. Repeated signals cannot interrupt cleanup, but blocked transport can delay it. Successful SIGTERM cleanup exits 143; failed stops retain resources for retry. Submitted stops do not prove controller receipt or physical halt. See ADR-0020.

These guarantees cover eval, rand, zero, and validate. Validation owns environment, abort watcher, and telemetry in one cleanup scope; it closes the environment before telemetry and releases ROS only after successful close. Failed closes retain resources for retry; completed closes are not repeated. ROS starts immediately before environment construction, not for dry runs or earlier preflight failures. Ctrl-C/SIGTERM during rollout propagates after cleanup without promising an interrupted-rollout report. Ordinary failures retain the invalid-evidence report and exit 4 if cleanup succeeds; persistent cleanup failure raises with its retry owner. Signal ownership covers preflight resources. A report does not prove shutdown.

Embedding processes must invoke this CLI runtime from the main thread. They may supply an initialized default ROS context only with native signal handlers disabled (SignalHandlerOptions.NO); conflicting handlers are rejected before hardware construction. Borrowed contexts remain active; caller Python signal handlers are restored on exit. Library-created RealEnv keeps caller-owned process signal and context handling.

srb real_agent ros accepts flat policy vectors on the environment’s action/raw topic, using generated action order and hardware mapping without simulator configuration. Missing, invalid, or stale input aborts via hardware stop methods. See the real ROS input contract for message bounds, startup and freshness timeouts, and cleanup behavior.

Nonhuman real_agent collect sources (random, zero, or a loaded policy) use RealEnv scalar steps but retain single-environment batch axes and the real split. Success-only collection uses terminal-step boolean success metadata when no success term exists; a collector time limit does not promote intermediate success to a final outcome. Optional generated COMMAND_MAPPING metadata records simulator command semantics and validates named action destinations before hardware acquisition. Native direct teleop and human collection require it, support ROS (default), SpaceMouse, and haptic input, and preflight the mapping and devices before hardware starts. Keyboard/gamepad and native policy-command teleop are unavailable; teleop --algo or --model is rejected. Policy evaluation and policy collection retain their existing routes.

Human collection retains the same dataset axes and real split. ROS and haptic have no manual episode controls, so environment termination or configured horizon completes an episode. The session closes input before releasing the environment and ROS context. Checked-in generated profiles still lack direct-command declarations; action dimensions and scales cannot supply one. Review live regeneration and deployment consequences before updating them. See the generation contract and native input details.

4. Advanced Workflows and Use Cases

Debugging with Zero and Random Agents

Only after hardware acceptance and operator-controlled bring-up may the zero and rand agents be used to exercise the configured action route. zero sends zero policy input, which can still actuate hardware after action mapping; it is not an emergency stop. rand sends random actions and must not be used as a safety check.

srb real_agent zero --env waypoint_navigation

srb real_agent rand --env waypoint_navigation

Fine-Tuning on Real Data

Note: Dreamer’s real-data fine-tuning path uses srb real_agent train --algo dreamer --continue and remains experimental.

5. Additional Task Examples

5.1 Excavation

The excavation task is designed for a Husky + Kinova mobile manipulator equipped with a scoop end-effector payload. Two variants are intended, depending on the motion-generation stack used on the real robot.

Construction is blocked. The shipped excavation routes currently fail construction checks. The commands illustrate adapter wiring only, not hardware readiness.

OSC (operational-space control) variant:

srb real_agent eval --env excavation --algo skrl_ppo \
  --hardware ros_depth_heightmap ros_tf ros_imu ros_kortex_cartesian ros_scoop_target ros_mw

skrl_ppo_rnn uses the SRB recurrent model and sequence-memory contract; its CPU/Isaac train/reload/eval smoke is in ADR-0008. These commands prove adapter wiring only, not hardware readiness.

Interfaces:

• ros_depth_heightmap — Projects RealSense depth into a local heightmap.
• ros_tf — TF pairs; OptiTrack pose proprio/base_pose_world_optitrack needs omitted ros_optitrack.
• ros_imu — Base orientation and angular rates.
• ros_kortex_cartesian — Kinova Cartesian admittance, FK, and contact force.
• ros_scoop_target — Excavation scoop-target observations.
• ros_mw — Pause/resume services.

Joint-velocity variant:

srb real_agent eval --env excavation --algo skrl_ppo \
  --hardware ros_depth_heightmap ros_tf ros_imu ros_kortex_joint_vel ros_scoop_target ros_mw

Additional interface:

• ros_kortex_joint_vel — Kinova joint-velocity commands; use with external motion generation (for example, NVIDIA fabrics) translating policy outputs into reactive targets.

Monitor success_rate, terminal_reward, particle_count_in_scoop_volume, and action_smoothness.

5.2 Terrain Landscaping

Canonical landscaping uses RaphRover + prismatic-velocity RaphShovel and one external heightmap. It has no onboard depth camera, IMU, blade-pitch channel, previous-action feedback, or Leo profile. Legacy Leo adapters remain under hyperparams/validation/terrain_landscaping_leo.yaml; they use separate task IDs and are not generated below.

Two distinct task IDs have generated modules; terrain_landscaping_mound is simulation-only:

Never relabel the general task with the crater hash. It owns a hashed per-episode mission manifest and requires it at deployment (REQUIRES_RUNTIME_TARGET_MANIFEST).

# Regenerate the checked-in bridge modules (writes the module in place)
srb real_agent gen --env terrain_landscaping_crater
srb real_agent gen --env terrain_landscaping

# Verify the checked-in modules are byte-identical to a fresh render
srb real_agent gen --env terrain_landscaping_crater --check
srb real_agent gen --env terrain_landscaping --check

Both commands run unqualified. A prior ExternalHeightmapFrameError from the general task’s undersized containment surface and heightmap overflow is fixed in the task; do not reintroduce a particle-height override.

For --check behavior, including ALL, see the freshness gate.

Generated IO contract. The action has three normalized velocity values: robot/cmd_vel on [0:2] and payload/joint_vel on [2:3]. Seven bounded observation leaves total 1287 float32 in contract order: current/target global and local heightmaps, proprio/mission_pose, proprio/base_velocity, and proprio_dyn/heightmap_age_s. The assembled space exposes four map outputs and one packed vector. Keep target_map_sha256, reset_layout_sha256, manifest_sha256, and io_schema_fingerprint distinct.

Deployment remains gated. Before ROS acquisition, hardware start, or policy load, canonical landscaping RealEnv checks capability tags claimed by supplied interfaces/providers. Required tags:

• raph.drive_velocity
• raph.shovel_prismatic_velocity
• landscaping.external_heightmap_batch
• landscaping.observable_task_evaluator

Only srb.interfaces.sim_to_real.validation.landscaping.LandscapingRealEvaluator claims landscaping.observable_task_evaluator. Pass it as RealEnv(..., task_evaluator=...): it grades accepted external frames against the verified target manifest, owns reward, and rejects mismatched task ID, IO-schema fingerprint, target manifest, reset layout, or normalization policy before ROS, hardware, or inference.

Missing capability tags raise DeploymentNotReadyError. Before deployment, verify Raph drive/shovel and mapper topics, message types, units, signs, saturation, watchdogs, acknowledgements, and measured joint-state feedback.

Simulation and fake-adapter tests do not prove hardware readiness; landscaping still requires lab evidence.

The canonical validation spec is hyperparams/validation/terrain_landscaping_crater.yaml. It declares no hardware, so running it fails closed on the capability preflight (exit 4) until the live adapters exist. Its observable metrics are success_rate, final_grading_mae_m, final_fraction_within_tolerance, action_smoothness, and map_age_s.

For learner readiness and observation gates, see Terrain Landscaping Training Readiness.

6. Validation Workflow

Advanced / reference. Sections 1–3 cover deployment and evaluation. Use this optional harness for repeatable release validation, drift tracking, and dashboards.

A structured validation session after srb real_agent eval records pass/fail, Rerun and JSONL telemetry, optional W&B data, and a cross-release dashboard.

6.1 Capture a sim baseline

Capture one sim baseline per checkpoint as an MP4:

srb agent eval --env excavation --algo skrl_ppo --model <ckpt> \
    --video env.num_envs=1

--video writes episode MP4s under the run directory’s videos/<timestamp>/. The validation spec does not configure this simulation command. For hardware, set n_episodes and .rrd in the spec; srb real_agent validate --wandb opts into W&B. The run ID supplies summary scalars; .rrd supports twin replay.

6.2 Validation spec YAML

The YAML binds the operator, generated RealEnv, and harness: task, policy, adapters, criteria, telemetry, baseline, and drift settings. CLI flags override only selected runtime fields.

task: excavation
algo: skrl_ppo
checkpoint: REQUIRED_AT_RUNTIME
hardware:
  - ros_depth_heightmap
  - ros_tf
  - ros_imu
  - ros_kortex_cartesian
  - ros_scoop_target
  - ros_mw
metrics_to_track:
  - success_rate
  - terminal_reward
  - action_smoothness
thresholds:
  success_rate_min: 0.5
  sim_real_ratio_min: 0.7
  action_smoothness_max_ratio: 1.5
  crash_rate_max: 0.0
  drift_z_max: 3.0
sim_baseline:
  wandb_run_id: entity/project/run-id
  local_rrd: baselines/excavation/<sha>.rrd
rerun:
  app_id: srb-validation
  web_port: 9090
n_episodes: 10
drift_window: 5
storage_root: logs/real_validation
notes: excavation release gate

Top-level fields:

Threshold fields:

Rerun fields:

Simulation baseline fields:

6.3 Run validation on the real robot

srb real_agent validate \
    --spec hyperparams/validation/excavation.yaml \
    --checkpoint <real-or-shared-ckpt> \
    --episodes 10 \
    --storage-root /tmp/srb_validation

Flags: --spec, --checkpoint, --episodes, --wandb, --no-tensorboard, --no-rerun-web, --storage-root, --post-status, --dry-run. Artifacts and TensorBoard are local by default; W&B is opt-in. Dry runs are non-hardware and never pass; see the real-validation CLI reference.

To request a graceful abort, create the exact file path printed by the running session at information log level, named .abort in its unique workspace beside the private .staging directory:

touch '/exact/session/workspace/.abort'

Programmatic callers can read ValidationHarness.abort_request_path; enable information logs to print it. The harness checks between reset, policy, and step calls, latches external_abort, stops future episodes, and records an incomplete failed session; current operations may finish first. This is not an emergency stop or watchdog. Cleanup removes the excluded request file. Inherited SRB_ABORT=1 cancels; exporting it in another shell cannot affect a running process.

The harness opens Rerun at http://localhost:9090, runs the requested rollouts in generated RealEnv, writes artifacts below, and checks: (a) success floor plus sim/real ratios when a baseline exists; (b) no policy_nan, hardware_event, disk_full, or safety abort (sensor_stale, sensor_incomplete, calibration_mismatch, manifest_mismatch); (c) tracked-metric z-scores within thresholds.drift_z_max over the last K sessions.

Horizon endings are time_limit_truncation with final metrics, never wall-clock timeout. Out-of-MDP aborts (sensor_stale, sensor_incomplete, calibration_mismatch, manifest_mismatch, operator_abort) have separate reasons/counts and are not task endings; operator abort fails the success floor. Incomplete episode counts or any such abort (including SRB_ABORT as external_abort) fail (a), are reported, and are excluded from drift; safety aborts also fail (b).

For sim/real ratios, higher-is-better requires real >= sim - (1 - sim_real_ratio_min) * |sim|; lower-is-better requires real <= sim + (1 / sim_real_ratio_min - 1) * |sim|; smoothness uses action_smoothness_max_ratio. Positive baselines reduce to ordinary ratios; zero baselines fail (a) with a detail. Drift is optional/skipped with fewer than two prior sessions, no comparable metrics, or drift_window < 2. Post-rollout evaluation errors (bad/missing baseline data) publish an invalid, non-hardware report and exit 4. A configured baseline resolved through a degraded fallback fails (a) and exits 1; unreachable W&B is one such case. Live landscaping rejects stale, incomplete, calibration-mismatched, or manifest-mismatched frames as session aborts; these fail (a), safety aborts also fail (b). success_rate is real-only and need not be in the sim baseline.

Exits with the validation result code:

Only generated srb_real/<task> environments register and run; resolution failure raises and never falls back to simulation srb/<task>. Generation requires a deployment profile, and shipped specs do not claim hardware validation; see the validation environment rule.

6.4 Validation artifacts

Validation writes one session directory and two storage-root artifacts:

logs/real_validation/
+-- _index.jsonl
+-- index.html
+-- <task>/
    +-- <uuid>_<utc-timestamp>/
        +-- current.json
        +-- generations/
            +-- <generation-uuid>/
                +-- spec.yaml
                +-- episodes.jsonl
                +-- rollout.rrd
                +-- tensorboard/
                +-- summary.json
                +-- report.md
                +-- badge.json

Session artifacts:

Storage-root artifacts:

6.5 Cross-session dashboard

Each run refreshes index.html with session status and (task, algo, metric) trends.

6.6 Twin replay (debugging)

Compare a failed session with its sim baseline:

srb real_agent replay-twin \
    --session logs/real_validation/excavation/<uuid>_<ts>/generations/<generation-uuid> \
    --baseline baselines/excavation/<sha>.rrd

The viewer overlays time-aligned simulated ghost and real recordings by episode.

6.7 Recovering a partial session

If summary.json is missing, rebuild from spec.yaml and episodes.jsonl:

srb real_agent rebuild-summary logs/real_validation/excavation/<uuid>_<ts>/generations/<generation-uuid>

6.8 Safety reminder

The harness observes estop, joint-limit, and contact-force events for criterion (b); it does not enforce safety or replace operator responsibility, deadman switches, joint-limit watchdogs, or estop circuitry.

7. Creating Custom Hardware Interfaces

Add a HardwareInterface subclass under sim_to_real/hardware implementing start (initialize), apply_action (send commands), observation (read sensors), and close (cleanup). The system discovers it for --hardware.