`srb dataset` — Manage Dataset Artifacts

The srb dataset command inspects, validates, converts, previews, and publishes SRB dataset artifacts. It works with canonical SRB JSONL datasets, robomimic HDF5 datasets, LeRobot-style layouts, and real-validation telemetry converted into datasets.

Usage

srb dataset <action> [options]

Actions

Action	Purpose
`info`	Print the SRB dataset manifest summary, artifact hash, and action-schema summary as JSON.
`validate`	Validate the manifest and referenced files. Declared counts are always checked against physical lengths; use `--deep` for additional row/table structure checks and `--strict` to treat warnings as errors.
`compatibility`	Report dataset compatibility for learning, collection/eval, or publication. Add `--algo ALGO` for a focused report, or omit it for the matrix.
`export`	Convert between supported layouts: `srb_jsonl`, `robomimic_hdf5`, and `lerobot`.
`pull`	Download an SRB dataset artifact from Hugging Face Hub. Resolve by `--repo-id` or `--task`/`--embodiment`/`--action-mode` identity. Use `--dry-run` to inspect the plan without network access.
`push`	Upload or dry-run a Hugging Face dataset publication. Use `--dry-run` before network upload.
`card`	Write a Hugging Face `README.md` dataset card.
`replay`	Preview a canonical dataset as a summary or with Rerun via `--backend rerun`.
`from-validation`	Convert a real validation session into a canonical SRB dataset with `split=real`.

Event-camera dataset capture and validation commands are tracked in the event-camera status matrix because some paths are live while real-fixture and Phase B validation work remains gated. The parser exposes record-events, validate-events, capture-real-evk4, export-lerobot-v3, export-zenodo-bundle, export-raw-evt3, and import-raw-evt3, and export-rosbag; nested validate-events actions are stats, reconstruction, and downstream. For the downstream stub, --synthetic-smoke runs the no-real-data adapter contract, --write-label-template creates the task-specific sidecar skeleton, and --check-real-target validates the physical EVK4 EVENT_HDF5 plus the sidecar before the future train/eval harness consumes it.

Publication Format

Canonical srb_jsonl is the preferred SRB publication format. lerobot is also publishable when the optional data dependencies are installed. Framework-native robomimic_hdf5 remains useful for training, but srb dataset push rejects it until it is exported to a publication format.

Action Metadata

Datasets store actions as flat vectors for broad framework compatibility. When the source environment exposes grouped actions, SRB records metadata.action_schema with the ordered component names, paths, shapes, dimensions, and flat-vector offsets. srb dataset info surfaces this as top-level action_dim, action_components, and action_schema fields so the action embodiment is easy to inspect without parsing raw metadata. srb dataset export preserves this field across canonical SRB JSONL, robomimic HDF5, and LeRobot layouts, and srb dataset validate checks that the declared schema agrees with metadata.action_dim. LeRobot exports also copy it into meta/info.json as srb_action_schema, so the action map remains visible to tools that read the LeRobot metadata directly; deep validation rejects mismatches between that field and the SRB manifest. Generated Hugging Face dataset cards include a compact action-schema table when this metadata is present. srb dataset replay prints the schema with the summary backend and logs per-component action values under actions/<component> when using the rerun backend.

Validation always reconciles declared episode/sample/action counts with the physical JSONL, HDF5, or Parquet lengths. If a format reader such as h5py or the pandas Parquet backend is unavailable, count validation fails explicitly; it never reports an unchecked dataset as valid.

Examples

# Validate a canonical dataset thoroughly before publishing
srb dataset validate /data/srb_excavation --deep --strict

# Convert a robomimic dataset to canonical SRB JSONL
srb dataset export /logs/excavation/collect/datasets/demos.hdf5 \
  --output /data/srb_excavation_policy --format srb_jsonl

# Preview a dataset with Rerun and save the replay
srb dataset replay /data/srb_excavation_policy \
  --backend rerun --save /tmp/srb_excavation.rrd

# Inspect upload contents without network access
srb dataset push /data/srb_excavation_policy --dry-run

# Resolve a Hub repo from artifact identity and inspect the plan
srb dataset pull --task excavation --embodiment kinova_gen3 \
  --action-mode smooth_osc --dry-run

# Download an SRB dataset artifact by repo-id
srb dataset pull --repo-id space_robotics_bench/srb_excavation_kinova_gen3_smooth_osc \
  --output /data/srb_excavation_policy

# Turn a real validation session into a dataset
srb dataset from-validation logs/real_validation/excavation/<session> \
  --output /data/srb_excavation_real \
  --embodiment kinova_gen3 --action-mode smooth_osc

from-validation keeps real-world data in split=real. By default, sessions with a validation algorithm or checkpoint are labeled controller=policy. Supported controller labels are human, policy, random, zero, and real.

Keyboard shortcuts