Reward & Normalization Configuration

Tasks define reward terms in a RewardCfg subclass of BaseRewardCfg (or a domain-specific subclass). Retune a term with a Hydra override, for example env.reward.penalty_action_rate.weight=-0.2; inspect the task’s config class for defaults.

from srb.core.reward import (
    BaseRewardCfg,
    GaussianRewardTerm,
    LinearRewardTerm,
)

class RewardCfg(BaseRewardCfg):
    penalty_action_rate: LinearRewardTerm = LinearRewardTerm(
        weight=-0.5, limit=4.0
    )
    reward_target_approach: GaussianRewardTerm = GaussianRewardTerm(
        weight=4.0, scale=0.5
    )

Each step, advance_step_state() updates per-step task state once before extract_step_return() computes observations, rewards, and terminations from the updated state. Reward metrics are mapped to terms, summed, and optionally normalized.

Reward terms

Terms are exported from srb.core.reward; most are in srb.core.reward.terms, while TukeyRewardTerm is in srb.core.reward.normalize.

• LinearRewardTerm scales the absolute metric and clamps to [-limit, limit]; use ScaledRewardTerm for sign-preserving scaling.
• QuadraticRewardTerm emphasizes larger errors. Offset linear and quadratic terms apply only beyond a deadband.
• GaussianRewardTerm gives a bell-shaped reward. Gaussian tracking terms follow a target; Gaussian zone tracking adds a tolerance band.
• HyperbolicBarrierRewardTerm and GaussianBarrierRewardTerm shape reward near safety thresholds such as ground clearance or joint limits.

Normalization

Bounds come from the signed reward_bounds of each term plus normalize_offset_per_step. One-sided terms contribute according to weight sign; sign-preserving terms such as ScaledRewardTerm and SmoothMaxRewardTerm contribute to both bounds.

R_worst and R_best are the summed worst and best term bounds plus their respective offsets. Large limits can make smaller terms negligible. Inspect the active bounds with:

worst, best = YourTaskRewardCfg().get_bounds()
print(f"Worst per-step: {worst}, Best per-step: {best}")