Event-Camera Status Matrix
Canonical readiness view for the synthetic IMX636 / EVK4 event-camera pipeline. Status terms on this page are the source of truth for the overview, mirror contracts, and dataset CLI reference.
| Area | Status | Current contract | Gate to next state |
|---|---|---|---|
| Sensor config | Live (scoped) | BaseEnvCfg._add_sensors materializes each EventCameraCfg into the active Isaac scene as a native RGB CameraCfg under the same logical name and keeps the original Isaac-free config in the scene-side event-camera bindings; DirectEnv binds those sensors after the scene is built. | A provider-backed fidelity differential with a concrete backend; the scoped local Isaac run used a strict test backend, not v2e, Metavision, or V2CE. |
| Task adoption | Live (scoped) | Opt-in task toggles retain EventCameraCfg and active-sensor intent; the runtime observation assembly discovers each bound sensor and registers its event representation plus event-count observations, merged into the flat observation every step. | A provider-backed delivery differential on a concrete backend, plus long/high-rate rollout evidence, before task adoption is called live beyond the scoped runner. |
noop backend | Live | Built-in smoke adapter emits zero events. | None. |
v2e backend | Live (non-streaming) | Canonical default backend; install with srb[event-v2e]. Upstream returns complete event arrays, so the adapter declares no streaming support and strict_streaming=True rejects it before construction. | Live optional-extra smoke with the private SuperSloMo mirror; public mirror when release-ready. |
metavision backend | Live (non-streaming) | Optional adapter and import contract are pinned; the provider materializes complete results, so it is rejected under strict_streaming=True. | Live optional-extra CI with OpenEB installed. |
v2ce backend | Live (non-streaming) | Optional adapter supports package-style V2CE providers and the script-style V2CE-Toolbox layout; it also materializes complete results and is rejected under strict_streaming=True. | Promote the opt-in V2CE checkpoint smoke into routine heavy CI; public mirror when release-ready. |
| Synthetic recording | Live | srb dataset record-events writes canonical EVENT_HDF5 episodes and events_manifest.jsonl. The srb agent <verb> --record-events surface materializes, binds, and records the sensor, but no concrete provider ships in the default container image, so Isaac rollout recording is not proven end to end with v2e, Metavision, or V2CE. | Install a concrete provider in the release image and retain a rollout differential; paired RGB MP4 output remains an optional provenance field, and no provider fidelity pass is claimed. |
| Statistical validation | Live | srb dataset validate-events stats compares rate, polarity, ISI, hot-pixel, and sparsity metrics. | Reference Gen4 fixture for routine CI. |
| Reconstruction validation | Live | srb dataset validate-events reconstruction provides baseline SSIM scoring and an E2VID-gated adapter path. | Promote the opt-in E2VID checkpoint smoke into routine heavy CI. |
| LeRobot export | Live | srb dataset export-lerobot-v3 wraps EVENT_HDF5 runs with an event_camera modality. | Publication smoke against a hosted dataset. |
| Zenodo / RAW / ROS interop | Live | export-zenodo-bundle, export-raw-evt3, import-raw-evt3, and export-rosbag preserve the canonical event run layout. | Heavy optional-extra smoke for RAW and ROS bag writers plus a licensed fixture mirror. |
| SuperSloMo weights | Private mirror | hf-andrejorsula-supersloMo is registered; fetching requires HF repo permission plus HF_TOKEN or hf auth login. | Make the HF repo public when release-ready. |
| E2VID weights | Private mirror | hf-andrejorsula-e2vid is registered; fetching requires HF repo permission plus HF_TOKEN or hf auth login. | Make the HF repo public when release-ready. |
| Gen4 reference fixture | Private mirror | hf-andrejorsula-active-marker-gen4 is registered; fetching requires HF dataset permission plus HF_TOKEN or hf auth login. | Confirm Prophesee redistribution terms before public release; downstream harness still needs a labelled/task-specific fixture. |
| Preset retune | Phase A | Physical-ordering and distinguishability invariants are tested. | Phase B calibration needs srb dataset capture-real-evk4 data plus SuperSloMo weights. |
| Release smoke | Phase B opt-in | Optional-extra names are tested without heavy dependencies; tests/integration/test_event_camera_phase_b_smoke.py validates the private active-marker fixture plus E2VID and V2CE checkpoints when explicitly enabled. | Promote to routine CI after public/licensed mirrors and a heavy release image are ready. |
| Downstream validation | Gated (CLI rejects) | Only --synthetic-smoke runs. Every other srb dataset validate-events downstream invocation, including --write-label-template and --check-real-target, is refused by the CLI preflight before any output is written because no validated train-on-synth / eval-on-real harness or model adapter exists; the scorecard, label-template, and real-target helpers exist in-tree and are unit-tested. | Land the real harness with a labelled/task-specific Gen4 target and model adapter, then open the preflight gate. |
| EVK4 capture | Gated (CLI rejects) | srb dataset capture-real-evk4 parses its arguments and then exits through the CLI preflight before creating any directory, manifest, or protocol output: no validated hardware capture backend and device contract exists in this build. | Requires a validated capture backend, hardware capture campaign, and publication approval. |
Status Terms
| Term | Meaning |
|---|---|
| Live | The interface is implemented and usable from SRB today, subject to any documented optional extra. |
| Live (scoped) | The path is implemented and wired, and a focused local Isaac run exercised it with a strict test backend. It is not a provider-fidelity, long-run, real-data, or hardware claim. |
| Live (non-streaming) | The backend adapter works, but the upstream provider materializes complete event results, so strict_streaming=True rejects it. |
| Gated (CLI rejects) | The command parses, then exits through a preflight capability gate before producing output. |
| Placeholder | The command or registry exists, but the default artifact intentionally refuses to download because SRB does not redistribute the data. |
| Gated | The interface is documented so downstream work can target it, but meaningful use waits on an explicit external artifact, hardware campaign, or harness. |
| Private mirror | A stable mirror is registered, but the HF repo is private and requires collaborator access plus an HF token. |
| Phase A | Lightweight invariant or contract is landed. |
| Phase B opt-in | Data-backed smoke exists in-tree, but runs only when explicitly enabled with local credentials/artifacts. |
| Phase B | End-to-end data-backed validation still waits on the task-specific artifacts or heavy CI image named in the row. |
User Paths
| Goal | Command | Status |
|---|---|---|
| List registered event-generator backends | srb event-camera backends --detailed | Live |
| List registered checkpoint mirrors | srb event-camera list-mirrors | Live |
| Fetch checkpoint by explicit URL | srb event-camera fetch-weights --url <url> --sha256 <sha> | Live |
| Fetch checkpoint by mirror | srb event-camera fetch-weights --mirror <short_name> | Private mirror; requires HF token |
| List registered fixture mirrors | srb event-camera list-fixtures | Live |
| Fetch fixture by explicit URL | srb event-camera fetch-fixture --url <url> --sha256 <sha> | Live |
| Fetch fixture by mirror | srb event-camera fetch-fixture --mirror <short_name> | Private mirror; requires HF token |
| Record synthetic event-camera data | srb dataset record-events --output OUT | Live |
| Run statistical fidelity validation | srb dataset validate-events stats --synth SYN --real REAL --output OUT | Live |
| Run reconstruction fidelity validation | srb dataset validate-events reconstruction --events EVENTS --rgb-npy RGB --times-npy TIMES --output OUT | Live |
| Run downstream fidelity validation | srb dataset validate-events downstream --synth-events SYN --real-events REAL --output OUT | Rejected by preflight; no harness |
| Run synthetic downstream adapter smoke | srb dataset validate-events downstream --synth-events SYN --task event_flow --output OUT --synthetic-smoke | Synthetic contract smoke live; no real metrics |
| Check downstream real-target labels | srb dataset validate-events downstream --synth-events SYN --real-events REAL --task event_flow --output OUT --check-real-target | Rejected by preflight; contract helper exists in-tree |
| Write downstream label template | srb dataset validate-events downstream --synth-events SYN --real-events REAL --task event_flow --output OUT --write-label-template | Rejected by preflight; contract helper exists in-tree |
| Export to LeRobot v3 | srb dataset export-lerobot-v3 EVENT_RUN --output OUT | Live |
| Build a Zenodo bundle | srb dataset export-zenodo-bundle EVENT_RUN --output OUT | Live |
| Export Prophesee RAW EVT3 | srb dataset export-raw-evt3 EVENTS_H5 --output OUT | Live |
| Import Prophesee RAW EVT3 | srb dataset import-raw-evt3 EVENTS_RAW --output OUT_H5 | Live |
| Export ROS bag | srb dataset export-rosbag EVENTS_H5 --output OUT | Live |
| Capture real EVK4 data | srb dataset capture-real-evk4 --task TASK --policy POLICY --output OUT | Rejected by preflight; no capture backend |
The opt-in Phase-B smoke is:
SRB_EVENT_CAMERA_PHASE_B_SMOKE=1 \
SRB_EVENT_CAMERA_FETCH=1 \
pytest -q tests/integration/test_event_camera_phase_b_smoke.py
For architectural rationale, see
docs/adr/0002-event-camera-sensor.md (ADR-0002: event-camera sensor design).