Skip to main content

Module storage

Module storage 

Expand description

The storage seam: the only thing workers coordinate through.

There is no coordinator process in atune. Every worker — a thread, a process running atune run, a SLURM array task — talks to one Storage and to nothing else. Everything that makes that safe is a property of this trait — the guide’s Storage is the prose version:

  • trial numbers are contiguous and race-safe, so seeds derived from them are too;
  • state changes go through Storage::try_transition, which reports a lost race as Ok(false) rather than an error;
  • finished trials are immutable — which is why a failing worker records its error text with Storage::set_error before it transitions the trial to TrialState::Failed, and why that text then survives;
  • reads are incremental: Storage::sync returns only what happened after a Cursor, which is what keeps the study loop from re-reading the whole study on every decision;
  • side-channel state is typed and versioned (StateBlob), not a pickled-object junk drawer.

Every backend passes one shared conformance suite: the conformance module, exported by the crate feature of the same name, generates one test per invariant for any backend that hands it a factory. MemStorage is the reference implementation and the oracle the suite is calibrated against.

Re-exports§

pub use failover::FailedOver;
pub use failover::FailoverPolicy;
pub use failover::fail_over_stale;
pub use finalization::FinalizationAcquireRequest;
pub use finalization::FinalizationObservation;
pub use finalization::FinalizationToken;
pub use lifecycle::AbandonRequest;
pub use lifecycle::ClaimRequest;
pub use lifecycle::ClaimTarget;
pub use lifecycle::FailStaleRequest;
pub use lifecycle::FencedMutation;
pub use lifecycle::FencedMutationRequest;
pub use lifecycle::FinalizedStateRequest;
pub use lifecycle::LeaseObservation;
pub use lifecycle::ParamWrite;
pub use lifecycle::RenewRequest;
pub use lifecycle::ReservationOutcome;
pub use lifecycle::ReserveAndCreateRequest;
pub use lifecycle::ReserveOutcome;
pub use lifecycle::TerminalOutcome;
pub use lifecycle::TerminalRequest;
pub use lifecycle::TrialLifecycleStorage;
pub use lifecycle::TrialOwnerToken;
pub use lifecycle::TrialReservation;
pub use outbox::EffectIntent;
pub use outbox::FinalizationAdvanceRequest;
pub use outbox::FinalizationIntent;
pub use outbox::FinalizationPhase;
pub use outbox::OutboxAdvanceRequest;
pub use outbox::OutboxEffect;
pub use outbox::OutboxEffectCompletion;
pub use outbox::OutboxEffectIntent;
pub use outbox::OutboxEffectKey;
pub use outbox::OutboxKey;
pub use outbox::OutboxPhase;
pub use outbox::OutboxRecord;
pub use outbox::REPLACEMENT_BLOB_KIND;
pub use outbox::REPLACEMENT_BLOB_VERSION;
pub use outbox::ReplacementOutcome;
pub use outbox::ReplacementSpec;
pub use outbox::replacement_scope;
pub use outbox_page::DEFAULT_OUTBOX_PAGE_MAX_BYTES;
pub use outbox_page::DEFAULT_OUTBOX_PAGE_MAX_RECORDS;
pub use outbox_page::OutboxCursor;
pub use outbox_page::OutboxPage;
pub use outbox_page::OutboxPageItem;
pub use outbox_page::OutboxPageRequest;

Modules§

conformance
The shared Storage conformance suite — “write your own backend”.
failover
Backend-independent stale-worker fail-over.
finalization
Durable, study-wide ownership for terminal finalization.
lifecycle
Atomic trial ownership and lifecycle capability.
lifecycle_conformance
Backend-independent conformance checks for super::TrialLifecycleStorage.
outbox
Durable terminal-finalization intent types.
outbox_page
Bounded, stateless traversal of durable finalization outboxes.

Structs§

Cursor
A position in a study’s change log.
ImportBatch
One serializable, fully staged import.
ImportBatchResult
Result of one atomic import publication.
ImportStudy
One study staged for import.
ImportTrial
One trial staged for import.
JournalOptions
Everything a JournalStorage can be tuned with.
JournalStorage
A study in one append-only file: grep-able, resumable, and shared between processes with no server.
MemStorage
The reference in-memory Storage backend: RwLock-guarded, copy-on-write reads.
SnapshotRequest
A request for one hydrated storage read.
StateBlob
A typed, versioned piece of state persisted through storage.
StorageSnapshot
Trial deltas and side state read for one study observation.
StudySummary
The stable identity data exposed by the optional study catalog.
SyncPage
One bounded page of Storage::sync_limited.

Enums§

Durability
When the log asks the filesystem to make a publication durable.
ImportOutcome
Terminal state represented by a staged import trial.
ImportTrialOutcome
Outcome for one staged trial in an ImportBatchResult.
Scope
What a StateBlob belongs to.
SnapshotConsistency
The consistency guarantee attached to a StorageSnapshot.
StudyCreateOutcome
The result of an atomic StudyCatalog::create_if_empty operation.
StudyLoadOutcome
The result of resolving a study name through StudyCatalog::find_by_name.

Traits§

AtomicImportStorage
Optional storage capability for all-or-none import publication.
Storage
The persistence seam.
StudyCatalog
Optional storage capability for stable study identity queries.