Environments – Mobile Manipulation

Environments combining mobility and manipulation capabilities.

Aerial Manipulation (_aerial_manipulation)

Template environment for aerial mobile manipulators. Configure via env.robot.mobile_base=<aerial_robot> and env.robot.manipulator=<manipulator>.

Compatible Robots

Generic Aerial Manipulator

Ground Manipulation (_ground_manipulation)

Template environment for ground mobile manipulators. Configure via env.robot.mobile_base=<ground_robot> and env.robot.manipulator=<manipulator>.

Compatible Robots

Generic Ground Manipulator

Orbital Manipulation (_orbital_manipulation)

Template environment for orbital mobile manipulators. Configure via env.robot.mobile_base=<orbital_robot> and env.robot.manipulator=<manipulator>.

Compatible Robots

Generic Orbital Manipulator

Mobile Debris Capture (mobile_debris_capture)

Capture of floating debris in microgravity with a floating-base mobile manipulator

Compatible Robots

Generic Orbital Manipulator

Termination semantics. distance_debris_too_far drives the penalty_debris_too_far reward and does not set terminated; the finite horizon is reported as truncated.

Spacewalk (spacewalk)

Humanoid spacewalk: rail-aligned waypoint traversal near spacecraft structure

Spacewalk casts microgravity humanoid traversal as a contact-rich locomotion problem: a thruster-less humanoid reaches a sequence of waypoints near a spacecraft and station-keeps at the final one, propelling itself by pushing off the structure and handrails as an EVA astronaut does (see Design notes for the physics). It ships in two modes — exterior EVA and interior IVA.

Use env.mode=eva for exterior ISS+Gateway, handrail-adjacent traversal (default), or env.mode=iva for inside-Kibou training. The base spacewalk env honours env.mode: setting env.mode=iva automatically retargets the waypoint path to the Kibou interior (IVA_WAYPOINT_PATH). The dedicated spacewalk_eva / spacewalk_iva aliases pre-select each mode and are the recommended way to keep the split explicit in benchmarks and launch configs. The two modes also differ substantially in traversal character — a multi-metre exterior leg with a lateral turn versus a short confined-space acquisition — as detailed under Locomotion below.

EVA and IVA share a single waypoint source of truth (EVA_WAYPOINT_PATH / IVA_WAYPOINT_PATH), from which the scenery handrails are also derived. (This automatic link applies to the built-in paths; a custom waypoint_path should be paired with a matching env.scenery.handrail_segment_specs, since the rails do not otherwise follow an overridden path.) The distance-to-nearest-handrail observation is always present, so the observation space is identical across modes — but it is a meaningful rail distance only in EVA. The IVA Kibou interior has no handrails, so there the value is the constant no_handrail_distance fallback — in IVA the agent cannot sense its propulsion surfaces (the module walls) and must push off reactively from IMU-sensed contact, which the confined interior keeps feasible.

Setting env.use_waypoint_path=false switches from path traversal to a single randomly placed target (within ±target_pos_range m of the spacecraft origin) — a simpler reaching variant useful for ablations.

The RL algorithm of record is DreamerV3 (hyperparams/dreamerv3.yaml); validation specs live at hyperparams/validation/spacewalk*.yaml. A policy must be trained first — no trained checkpoint ships yet:

# Train (DreamerV3 is the algorithm of record)
srb agent train --env spacewalk_eva

# Evaluate a trained checkpoint (reports the metrics below)
srb agent eval --algo dreamer --env spacewalk_eva --model <path/to/checkpoint>

Benchmark protocol (intended; no results ship yet — this is a draft). The protocol follows the committed configs so it is reproducible once policies are trained.

Training. DreamerV3 with num_envs = 16 parallel environments, world model deter = 512, stochastic latent 32 × 32, imagination horizon = 15, replay capacity 1e6, replay ratio = 512, up to max_iterations = 2e6 policy steps (all in hyperparams/dreamerv3.yaml). These are the shared SRB orbital-task defaults (the spacewalk block is identical to the excavation/terrain_landscaping blocks), not tuned for spacewalk’s harder contact-locomotion regime — expect to tune them when training in earnest. Repeat training over independent random seeds — supplied per run, since the committed configs do not pin a training seed — and report mean ± standard deviation.

Evaluation. n_episodes = 10 rollouts per checkpoint (hyperparams/validation/spacewalk*.yaml), reporting five metrics:

• success_rate — fraction of episodes that reached and held the final waypoint.
• min_distance_to_target — closest approach achieved to the current target waypoint over the episode, metres (small once any waypoint is neared).
• min_distance_to_final_target — closest approach to the final waypoint over the episode (a clean completion measure).
• waypoint_progress — normalised advancement along the path, index / (num_waypoints − 1).
• dist_to_nearest_handrail — body-to-rail distance (meaningful in EVA).

(The real-hardware validation spec’s metrics_to_track is a superset of this reported set: it additionally records the generic validation-harness metrics terminal_reward, action_smoothness, and wall_seconds.)

Interpreting near-zero success. Because contact-based locomotion is hard to discover, early policies may score near-zero success_rate; min_distance_to_target and waypoint_progress are the primary signals of partial competence and should be reported even when success is zero, rather than collapsing the result to a single binary rate. Read together they localise the bottleneck:

• low waypoint_progress with a large min_distance_to_target — the agent never gets close to even the first waypoint (push-off discovery fails);
• low waypoint_progress with a small min_distance_to_target — it nears a waypoint but cannot reliably advance;
• waypoint_progress near 1.0 with zero success_rate — the failure is on the final leg. min_distance_to_final_target then separates the two sub-cases: still above success_radius means the agent never reached the final waypoint, whereas near zero with zero success_rate means it reached the final waypoint but could not hold the station-keep (the terminal velocity-arrest).

For EVA, ISSGatewayScenery can be customized without editing code:

• env.scenery.station_template_usd_path — replace ISS+Gateway pair with a single USD station file.
• env.scenery.use_exact_station_mesh_collision — enable exact triangular mesh collision (default: false).
• env.scenery.add_handrails / env.scenery.handrail_segment_specs — control optional support rails aligned to the waypoint path.
• env.scenery.handrail_specs — legacy shorthand for X-axis rails.

An episode is reported as a success once the humanoid reaches the final waypoint (within success_radius = 0.5 m of it) and remains inside that radius for success_stability_steps = 10 consecutive control steps. The success flag is sticky: once set, it stays set for the rest of the episode, so the metric counts “reached and briefly held the goal”, not “ended the episode at the goal”.

Each episode resets the humanoid within a ±0.5 m box (at the EVA path start, or about the Kibou origin for IVA) with fully random orientation and a small random velocity (±0.1 m/s linear, ±5°/s angular) — so a policy cannot assume a fixed start pose or a zero initial drift. Physics and control run at the same 25 Hz (decimation 1, dt = 40 ms), so the push-off contact dynamics the locomotion relies on are resolved at the relatively coarse control step — relevant to both reproduction fidelity and the difficulty of learning stable contacts.

Observation & action spaces. The policy observes (DreamerV3 vector inputs): robot orientation as a 6D rotation (tf_rot6d_robot), the robot→target-waypoint vector in the robot’s body frame (tf_pos_robot_to_target, so the target is seen egocentrically), scalar distances to the spacecraft origin (dist_to_spacecraft) and to the nearest handrail (dist_to_nearest_handrail), normalised traversal progress (waypoint_progress), body-frame linear and angular velocity, joint positions and velocities, and IMU linear-acceleration / angular-velocity. The action is per-joint position targets for the selected humanoid; the action dimensionality equals the robot’s actuated-joint count, resolved at runtime from the USD articulation. Targets are bounded to each joint’s physical position limits (JointPositionToLimits), so the policy cannot command out-of-range configurations — unlike the raw-scale observations, the action space is intrinsically bounded. There is no explicit contact-state observation — the agent infers surface contact (needed to time push-offs, per the locomotion notes below) from the IMU’s linear acceleration; adding an explicit contact signal is a natural future-work lever.

Observations are fed at their raw scales (metre-scale distances alongside the [0,1] waypoint_progress and [-1,1] rot6d); the task applies no manual normalisation because DreamerV3’s built-in symlog encoding absorbs that dynamic range. A non-DreamerV3 algorithm should add its own observation normalisation.

Compatible Robots

Humanoid21 | Humanoid28 | Unitree H1 | Unitree G1

These are configuration-compatible embodiments, not validated publication baselines. Unitree G1 remains the tested publication default for the current Spacewalk draft.

Reward design. The nine terms compose into four functional groups: (1) long-range guidance — reward_distance_to_target (Gaussian, scale 2.5), reward_approach_velocity (direction alignment), and reward_progress (signed per-step delta) shape the approach from far out; (2) terminal acquisition — reward_reaching (sharp soft-threshold at a fixed 0.5 m reaching radius, independent of the configurable success_radius — they coincide only at the default success_radius = 0.5) becomes the dominant signal at the goal; (3) station-keeping — reward_stability (low-velocity Gaussian, gated to the success region) and reward_success_hold (streak fraction) reward holding the final waypoint; (4) regularisation and safety — the action-rate and angular-velocity penalties damp jittery actuation, and the one-sided penalty_drift_from_spacecraft keeps the agent within the working envelope — the ≤~7 m-from-origin region the waypoints span, beyond which the penalty’s 9 m activation knee deliberately sits so it never fires during legitimate traversal. Drifting off-structure is effectively unrecoverable (a thruster-less humanoid with no surface to push from cannot propel back), so the penalty discourages escape without pulling the agent toward the origin. The net effect is a monotonic landscape — total reward rises from drifting away, through productive traversal, to holding at the goal. Unit tests pin this landscape and check the reward resists common reward-hacking patterns: in-place spinning (defeated by the angular penalty, since the stability term gates on linear velocity only), toward/away oscillation, and hovering just outside the success boundary.

One shaping limitation is known and left as future work. The dense reward_distance_to_target and reward_reaching terms track the current waypoint, which teleports to the next leg the instant an intermediate waypoint is entered (and there is no explicit per-advance bonus — waypoint_progress is only observed/reported, never rewarded). Crossing an intermediate waypoint therefore causes a transient per-step reward drop of roughly the distance+reaching value of the leg just closed. The coarse landscape above still favours reaching and holding the final goal (the global optimum), but a farsighted agent must see past this local dip rather than camp just outside each intermediate radius. A characterisation test (test_intermediate_waypoint_advance_has_known_reward_cliff) pins the current magnitude; the natural fix — folding waypoint_progress into the reward or using a potential-based path-following term — changes the reward landscape and so needs GPU re-validation before it ships.

Design notes. Episodes are fixed-horizon (20 s / 500 steps) with truncation only — there is no hard failure termination, so every rollout runs the full horizon. The horizon is a truncation, not a terminal (termination is always false in the step return): a correct agent bootstraps its value estimate at the time limit rather than treating reaching it as a failure — important to handle right when reproducing with a non-DreamerV3 algorithm. The reward shapes traversal alone: there is no contact or collision penalty, even though the station, Gateway, and handrails are rigid colliders. An agent is therefore neither rewarded for grasping a handrail nor penalised for brushing the structure — the benchmark scores rail-adjacent waypoint traversal and station-keeping, not contact-aware EVA. Natural future-work extensions: a collision penalty targeting high-impact contact (distinct from the controlled push-off the agent relies on to move — penalising all contact would break locomotion), an explicit handrail-grasp objective, and an explicit contact-state observation.

Locomotion. The agent commands only humanoid joint positions — there are no thrusters and no ground in the microgravity scene. Internal joint motion cannot change the body’s centre-of-mass velocity (momentum is conserved), so to translate toward distant waypoints the humanoid must push or pull against the station and handrails, as an EVA astronaut does; the small randomised initial drift (±0.1 m/s) covers only a fraction of a multi-metre leg. This gives the handrails a triple role — distance cue, propulsion surface, and (the EVA path terminates at a rail) a brace for arresting velocity to station-keep at the goal. It is also why structure contact is left unpenalised, and it makes the benchmark a hard contact-rich locomotion problem — part of why no policy is trained yet. The reward credits the outcome of locomotion (approach velocity, progress, distance), not the contact event itself, so push-off must be discovered from its downstream effect rather than followed up a local gradient; densifying that signal with a contact/grasp-shaping term (the future work noted above) is the natural way to make the task more learnable.

EVA and IVA also differ in character: EVA is a multi-metre exterior traversal along handrails with a lateral turn, whereas IVA is a short confined-space task inside Kibou. The ~12.5 m EVA path within the 20 s horizon implies a ~0.6 m/s required average, so push-off chaining must be efficient — there is little slack for dawdling before the episode truncates. Because the shared success_radius (0.5 m) is comparable to the interior waypoint spacing, IVA mainly exercises final-waypoint acquisition and station-keeping in a confined volume rather than long-range traversal.

Spacewalk Eva (spacewalk_eva)

Humanoid EVA traversal along exterior spacecraft handrails

Compatible Robots

Humanoid21 | Humanoid28 | Unitree H1 | Unitree G1

Spacewalk Iva (spacewalk_iva)

Humanoid IVA traversal inside the Kibou module

Compatible Robots

Humanoid21 | Humanoid28 | Unitree H1 | Unitree G1