Trait Storage 

pub trait Storage: Send + Sync {
    // Required methods
    fn create_study(&self, cfg: StudyConfig) -> Result<StudyId>;
    fn study_config(&self, study: StudyId) -> Result<StudyConfig>;
    fn create_trial(
        &self,
        study: StudyId,
        template: Option<TrialTemplate>,
    ) -> Result<TrialRecord>;
    fn try_transition(
        &self,
        trial: TrialId,
        from: TrialState,
        to: TrialState,
    ) -> Result<bool>;
    fn write_params(
        &self,
        trial: TrialId,
        batch: &[(String, Distribution, ParamValue)],
    ) -> Result<()>;
    fn report(&self, trial: TrialId, step: u64, values: &[f64]) -> Result<()>;
    fn set_values(&self, trial: TrialId, values: &[f64]) -> Result<()>;
    fn set_error(&self, trial: TrialId, message: &str) -> Result<()>;
    fn heartbeat(&self, trial: TrialId, now_millis: u64) -> Result<()>;
    fn put_state(&self, scope: Scope, blob: StateBlob) -> Result<()>;
    fn get_state(&self, scope: Scope) -> Result<Option<StateBlob>>;
    fn sync(
        &self,
        study: StudyId,
        cursor: Cursor,
    ) -> Result<(Vec<TrialDelta>, Cursor)>;
    fn get_trial(&self, trial: TrialId) -> Result<FrozenTrial>;

    // Provided methods
    fn backing_paths(&self) -> Option<Vec<PathBuf>> { ... }
    fn study_user_attrs(&self, _study: StudyId) -> Result<UserAttrs> { ... }
    fn set_study_user_attr(
        &self,
        _study: StudyId,
        _key: &str,
        _value: Value,
    ) -> Result<()> { ... }
    fn set_trial_user_attr(
        &self,
        _trial: TrialId,
        _key: &str,
        _value: Value,
    ) -> Result<()> { ... }
    fn read_snapshot(
        &self,
        request: &SnapshotRequest,
    ) -> Result<StorageSnapshot> { ... }
    fn trial_lifecycle(&self) -> Option<&dyn TrialLifecycleStorage> { ... }
    fn atomic_import(&self) -> Option<&dyn AtomicImportStorage> { ... }
    fn study_catalog(&self) -> Option<&dyn StudyCatalog> { ... }
    fn sync_limited(
        &self,
        study: StudyId,
        cursor: Cursor,
        max_trials: usize,
    ) -> Result<SyncPage> { ... }
}

The persistence seam.

Object-safe and Send + Sync: a study holds Arc<dyn Storage> and hands it to every worker.

Implementations must uphold the invariants documented on each method. They are not suggestions — samplers, schedulers and the determinism contract all rest on them, and the conformance suite (M1.2) asserts them.

Required Methods

fn create_study(&self, cfg: StudyConfig) -> Result<StudyId>

Creates a study and returns its identity.

Invariants

• Every study gets a distinct StudyId, whatever its name. Unique names are not required: the conformance suite creates two studies called the same thing, and a backend may accept or reject that.
• The configuration is stored verbatim — seed, directions, declared space, resource unit and metric names all come back from study_config unchanged. That is what lets a second process reproduce the first one’s sampling.
• A study’s StudyConfig is immutable after this call. There is no configuration-update operation in this trait; readers may therefore pair one study_config result with a later snapshot without a torn-config concern.

Errors

Error::Conflict if a study with the same name already exists and the backend enforces unique names; Error::InvalidSpace if the configuration does not validate. Note that StudyConfig validates on construction and on deserialization, so an invalid one cannot be handed to this method through any public API; a backend that re-validates is being defensive, not redundant.

fn study_config(&self, study: StudyId) -> Result<StudyConfig>

Reads a study’s configuration back.

The returned configuration equals the stored one, including the seed — which is what lets a second process reproduce the first one’s sampling. It is immutable after create_study, so a StudyReader may retain it while refreshing its observation.

Errors

Error::NotFound if the study does not exist.

fn create_trial( &self, study: StudyId, template: Option<TrialTemplate>, ) -> Result<TrialRecord>

Creates a trial, optionally pre-loaded from a template.

Invariants

• The returned TrialNumber is contiguous within the study: the n-th successful call returns n - 1.
• Numbering is race-safe: concurrent callers never receive the same number and never leave a gap. This is what makes seed derivation sound under parallelism.
• Numbering is per study: two studies both start at 0 and never interleave. TrialIds, by contrast, are distinct storage-wide.
• The trial starts in TrialState::Waiting.
• The template is honoured, not advisory. Every parameter of TrialTemplate::fixed appears in the created trial’s params with exactly the value written, and TrialTemplate::parent is persisted to FrozenTrial::parent. A template carries no distributions — those are unknown until sampling — so distributions stays empty and the fixed parameters do not enter the compatibility gate until a write_params supplies them.
• None and an empty template are equivalent, and neither is an error.

Errors

Error::NotFound if the study does not exist; a backend-specific Error::Storage otherwise. A parent naming a trial that does not exist is not validated in M1 — the reference falls back to recording it verbatim, since forks are an M4 feature and nothing resolves the link yet.

fn try_transition( &self, trial: TrialId, from: TrialState, to: TrialState, ) -> Result<bool>

Atomically moves a trial from from to to.

Invariants

• Returns Ok(true) if the trial was in from and is now in to.
• Returns Ok(false), never an error, when the trial was in some other state — that is a lost race, and losing a race is normal. Reserve errors for genuine faults.
• The current-state check comes first. A stale from is Ok(false) even when from -> to could never be legal: the caller has already lost the race, and what it would have asked for is then irrelevant. Error::Conflict is reserved for a caller whose from is current and whose to is illegal — a genuine bug, not a race.
• Rejects transitions out of a finished state: finished trials are immutable. A caller racing a worker that has already finished the trial therefore sees Ok(false), not an error.
• Legality is exactly TrialState::can_transition_to, including that a self-transition is never legal.
• Under concurrency exactly one caller wins; the losers get Ok(false) and the trial ends in the winner’s target state.
• A rejected or lost call leaves the trial untouched.

Errors

Error::NotFound if the trial does not exist; Error::Conflict if from is the trial’s current state but from -> to is not a legal transition; Error::Storage on a backend fault.

fn write_params( &self, trial: TrialId, batch: &[(String, Distribution, ParamValue)], ) -> Result<()>

Records sampled parameters together with the distributions they came from.

Batched on purpose: one relative-sampling step writes all of a trial’s parameters in a single round trip, which is what removes Optuna’s per-suggest chattiness.

Invariants

• A batch is all-or-nothing. One rejected entry rejects the whole batch, including the entries that precede it: after a failure the trial’s params and distributions are exactly what they were.
• An accepted batch writes every entry together with the distribution it came from, onto that trial.
• The compatibility gate is study-wide: a parameter name keeps the distribution kind, log flag and categorical choice set it was first recorded with, across trials. Writing an incompatible one fails with Error::Incompatible naming the offending parameter (see Distribution::is_compatible_with). Numeric bound drift is accepted — a define-by-run space may widen or narrow between trials — and the drifted distribution is recorded on the trial that used it. The gate does not extend across studies.
• Writing to a finished trial fails with Error::Conflict.
• An empty batch is accepted.

Errors

As above, plus Error::NotFound and — in the reference backend, which validates what it stores — Error::InvalidSpace for a malformed distribution and Error::OutOfRange for a value its own distribution does not contain.

fn report(&self, trial: TrialId, step: u64, values: &[f64]) -> Result<()>

Appends an intermediate report.

step is measured in the study’s declared ResourceUnit, and values is a slice because pruning is multi-objective-aware from day one.

Invariants

• Reports are retained in the order they were accepted, not sorted by step: a never-seen step is appended even when it is numerically earlier than one already recorded.
• Reports are keyed by step. Re-reporting a step already present overwrites that entry in place; it must not append a second one and must not move earlier steps behind later ones. This is what keeps FrozenTrial::last_intermediate meaning “the most advanced report accepted so far”, which the pruned-trial objective rule rests on.
• values is stored verbatim, however many there are and whether or not they are finite: a NaN report is a real observation about a diverging run, not something to reject or round. An empty values is accepted.
• Reporting to a finished trial fails with Error::Conflict.

Errors

As above, plus Error::NotFound.

fn set_values(&self, trial: TrialId, values: &[f64]) -> Result<()>

Records a trial’s final objective values.

Invariants

• The number of values must match the study’s direction count exactly; too few (including none) and too many are both rejected with an error. The reference backend uses Error::Conflict; the conformance suite requires only that it is an error.
• Accepted values are stored verbatim, non-finite ones included.
• The recorded values take precedence over the last intermediate when a trial is pruned (see FrozenTrial::objective_value).
• Setting values on a finished trial fails with Error::Conflict; the caller transitions the state after the values are durable, so a trial that reaches Complete already carries them.

Errors

As above, plus Error::NotFound.

fn set_error(&self, trial: TrialId, message: &str) -> Result<()>

Records why a trial failed.

The failure path is write the message, then transition to TrialState::Failed — in that order, because a finished trial is immutable and the error would otherwise be unrecordable. That the text survives is an invariant of the model, not a nicety (the states a trial moves through): failed trials are invisible to samplers, but their record, parameters and error text survive, because losing them throws away the most interesting data a tuning run produces.

Invariants

• The message survives verbatim into FrozenTrial::error, and is visible through both get_trial and sync.
• A later call replaces the message; it does not append.
• It marks the trial changed for the next sync.
• Recording an error on a finished trial fails with Error::Conflict — finished trials are immutable, and this method is no exception.

Errors

As above, plus Error::NotFound.

fn heartbeat(&self, trial: TrialId, now_millis: u64) -> Result<()>

Records that the trial’s owner is still alive at now_millis.

The timestamp is passed in rather than read, because atune_core never touches the system clock directly. Another worker may fail a trial over once its heartbeat is stale by the study’s grace period.

Invariants

• The caller’s timestamp is stored verbatim — a backend must never substitute a clock reading of its own, or the staleness comparison would be made against two different clocks. Any u64 is a legal stamp, 0 and u64::MAX included.
• A trial has no heartbeat until one is recorded.
• Legal in every non-terminal state (Waiting, Running, Paused).
• Beating a finished trial fails with Error::Conflict: a heartbeat is a write like any other, and finished trials are immutable.

Errors

As above, plus Error::NotFound if the trial does not exist.

fn put_state(&self, scope: Scope, blob: StateBlob) -> Result<()>

Writes the state blob of scope, replacing any previous one.

Invariants

• It replaces: it does not append or merge, and the blob’s kind and version are part of what is replaced.
• Scopes are independent keys. Writing one must not disturb any other, including a sibling scope of the same study and a Scope::Named slot whose name is empty or absurdly long.
• A state write is not a trial change, so it produces no delta and does not advance the study’s Cursor.

Errors

Error::NotFound if the scope’s owner does not exist; Error::Storage on a backend fault.

fn get_state(&self, scope: Scope) -> Result<Option<StateBlob>>

Reads the state blob of scope, if one was ever written.

Errors

Error::Storage on a backend fault. A missing blob is Ok(None), not an error.

fn sync( &self, study: StudyId, cursor: Cursor, ) -> Result<(Vec<TrialDelta>, Cursor)>

Returns everything that happened in study after cursor.

Invariants

• The returned deltas are strictly after cursor, in strictly increasing sequence order, and the returned cursor covers the last of them.
• The returned cursor is monotone: never smaller than cursor, and equal to it exactly when there is nothing new. A repeated sync at the head is therefore empty and idempotent.
• A cursor from the future is tolerated: no deltas, no error, and still no backwards cursor.
• Passing Cursor::BEGIN returns the whole study, and folding the increments reproduces that full re-read exactly.
• Every delta belongs to the study that was synced: one study’s sync never leaks another’s trials, and the sequence numbers are per study.
• The result contains one latest complete snapshot per trial changed after cursor. Several mutations to one trial before a reader syncs are deliberately coalesced; this is a latest-state feed, not mutation history. A trial’s delta sequence is the position of its latest included mutation, so gaps between returned sequence numbers are expected.

Errors

Error::NotFound if the study does not exist.

fn get_trial(&self, trial: TrialId) -> Result<FrozenTrial>

Reads one trial’s snapshot.

Invariants

• The snapshot agrees with the TrialRecord the trial was created with about identity, ordinal and initial state, and reflects every write that has been accepted for it since.

Errors

Error::NotFound if the trial does not exist.

Provided Methods

fn backing_paths(&self) -> Option<Vec<PathBuf>>

Local files that an exporter must never replace, including fixed sidecars.

None means provenance is unknown; callers must not infer that replacing an existing output is safe. Some(Vec::new()) explicitly identifies a backend with no local backing files. Paths must be absolute and captured at opening, not resolved against a later working directory. This additive capability performs no filesystem operations in the model layer.

fn study_user_attrs(&self, _study: StudyId) -> Result<UserAttrs>

Returns all user-owned JSON metadata attached to a study.

Old studies read as an empty map. Implementations must return a complete snapshot whose keys are deterministic and must never synthesize entries in the reserved atune: namespace.

Errors

Returns Error::NotFound if the study does not exist, or a backend-specific error if the metadata cannot be read. The default implementation returns Error::Unsupported.

fn set_study_user_attr( &self, _study: StudyId, _key: &str, _value: Value, ) -> Result<()>

Inserts or replaces one study user attribute.

The entire atune: prefix is reserved. Reusing a normal key replaces its previous JSON value atomically.

Errors

Returns Error::NotFound if the study does not exist, Error::Conflict for a reserved key, or a backend-specific error if the value cannot be stored. The default implementation returns Error::Unsupported.

fn set_trial_user_attr( &self, _trial: TrialId, _key: &str, _value: Value, ) -> Result<()>

Inserts or replaces one user-owned JSON attribute on an unfinished trial and marks that trial changed for sync.

Lifecycle-managed trials must use the fenced mutation exposed by TrialLifecycleStorage. The atune: prefix is reserved.

Errors

Returns Error::NotFound if the trial does not exist, Error::Conflict for a reserved key or a finished trial, or a backend-specific error if the value cannot be stored. The default implementation returns Error::Unsupported.

fn read_snapshot(&self, request: &SnapshotRequest) -> Result<StorageSnapshot>

Reads trial deltas, requested side state, and optional finalization records for one observation.

The default implementation is deliberately SnapshotConsistency::BestEffort: for each pass it makes one sync trial-list read, one get_state call per concrete scope, and, when the request includes an outbox and the lifecycle capability is present, finite TrialLifecycleStorage::list_outbox_page pages followed by one get_trial call per listed record. If the page capability is unsupported or a record exceeds its finite page budget, the source-compatible TrialLifecycleStorage::list_outbox fallback is used. The full snapshot remains potentially unbounded and retains SnapshotConsistency::BestEffort. Concrete scopes are the explicit scopes plus the per-returned-trial selectors; named prefixes cannot be enumerated by this fallback. This keeps third-party storage implementations source-compatible, but concurrent writes can appear between those calls. Built-in backends override this method and perform every component under one lock or transaction.

crate::study::StudySnapshot may make one second fallback pass when returned trial parameters reveal policy names absent from the immutable study configuration. It never repeats policy discovery indefinitely; both passes retain the SnapshotConsistency::BestEffort marker. A state-only write does not advance Cursor, so a second cursor read cannot upgrade this result into an atomic claim.

Errors

Any trial, state, or outbox read error from the backend.

fn trial_lifecycle(&self) -> Option<&dyn TrialLifecycleStorage>

Returns the backend’s atomic trial-lifecycle capability, if available.

This default keeps existing storage implementations source-compatible. A caller requiring ownership-safe lifecycle operations must treat None as Error::Unsupported, not reconstruct the operations from the raw CRUD methods.

fn atomic_import(&self) -> Option<&dyn AtomicImportStorage>

Returns the backend’s whole-import atomic-publication capability, if available.

This additive default keeps existing storage implementations source compatible. Import callers must treat None as Error::Unsupported, not emulate atomic publication with a sequence of ordinary CRUD calls.

fn study_catalog(&self) -> Option<&dyn StudyCatalog>

Returns the backend’s study-catalog capability, if available.

This default keeps existing storage implementations source-compatible. A caller requiring catalog identity operations must treat None as Error::Unsupported, not emulate lookup by probing guessed StudyId values.

fn sync_limited( &self, study: StudyId, cursor: Cursor, max_trials: usize, ) -> Result<SyncPage>

sync, bounded to at most max_trials deltas.

The point is work, not convenience: a transport that must put a sync result in one bounded buffer cannot ask for a whole dense study first and trim afterwards. A backend that can stop reading early overrides this; the default below is correct but reads everything, so it bounds only what the caller then has to hold.

Invariants

• Every sync invariant still holds for the page: the deltas are strictly after cursor, in increasing sequence order, one latest complete snapshot per included trial, and SyncPage::cursor is monotone.
• max_trials == 0 means no bound and is exactly sync.
• SyncPage::more is true only when the bound truncated the page. When it is false the page equals a full sync from the same cursor.
• Paging to exhaustion from Cursor::BEGIN folds to exactly what one unbounded sync returns: the cursor of a truncated page covers its last delta and no more, so the next page resumes at the first delta left out.

Errors

Error::NotFound if the study does not exist.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl Storage for JournalStorage

impl Storage for MemStorage