Storage

A study lives in storage, not in the handle you built it with. That is the single idea this page exists to install, because almost everything else follows from it: a study survives the process that created it, several workers can drive one study at once, and there is no coordinator process — storage is the only thing workers coordinate through.

What storage is responsible for

Storage is not a passive log. Five of its duties are correctness requirements for every backend that runs a study — the lifecycle methods default to Unsupported on a plain Storage — and they are what make coordinator-free workers safe:

• Trial numbers. Storage assigns them: contiguous from 0, per study, race safe. Two workers asking for a trial at the same instant get two different numbers with no gap and no duplicate. Seeds are derived from the number, so a gap or a repeat would break reproducibility, not just tidiness.
• Study identity, and the configuration under it. The seed, the directions, the declared space, the resource unit and the metric names come back exactly as they went in — which is what lets a second process reproduce the first one's sampling.
• Exactly one owner per running trial. Study execution uses the atomic lifecycle capability: reserve, claim, and renew return an owner/epoch token, and every ownership-scoped write checks that token. Raw CRUD retains its compare-and-set interface for compatibility, but Study never emulates an atomic claim with separate raw writes.
• Terminal work is recoverable. Terminal outcome, error text, seam state, and retry/requeue intent are committed with a durable idempotent outbox. Replaying completed work is a no-op.
• Trial budgets are global. Reserving and creating a trial under max_trials is one atomic storage operation. The limit counts durable trial records across handles and processes, including a record whose later startup fails.

Storage never reads the operating-system clock implicitly. Backends use an injected Clock for storage-owned timestamps, while caller timestamps remain explicit. Tests can therefore drive time deterministically. Lease expiry never compares wall-clock readings from two different workers.

Study names are deliberately not unique. Built-in backends expose the optional StudyCatalog capability for atomic create-if-empty, deterministic listing, and exact-name lookup; duplicate names return a typed conflict rather than an arbitrary identity. Consumers such as the CLI and Python layer their one-study-per-spec policy on that catalog. A third-party backend may omit the capability, but adapters then fail explicitly instead of probing numeric IDs. Storage also cannot promise durability beyond what the chosen backend provides.

One storage spec, one study

A storage spec names exactly one study in the CLI and the Python binding; there is no study registry, so atune run --study s.atj needs no --study-name. Four consequences:

• Creating a study in a non-empty storage is an error, atomically, so two racing creators cannot both win.
• Reopening is a separate verb: atune.load_study(spec) from Python, create_study(..., load_if_exists=True) for a script that both creates and resumes, StudyLocator plus builder().load(id) from Rust.
• A name is a check, not a lookup. Passing one verifies that the study in the spec is the study you meant, and mismatching says which name is actually there.
• There is no get_all_study_names and no delete_study, because there is nothing to enumerate and the file is the study. Removing a study is removing its storage.

One layer down, the Storage trait is multi-study (a StudyId addresses a study); the one-study rule is an adapter policy over the optional StudyCatalog capability. Python refuses a multi-study backend, and a backend without the catalog fails with a typed unsupported error. Migrating from a tool with a study registry starts at Migrate from Optuna.

The backends

Every backend answers the same questions the same way — the conformance suite further down this page is what holds them to it — so the choice between them is operational, never semantic. No backend gives you a different study; each gives you a different set of costs. The storage catalogue names each one's Rust type and the cargo feature that gates it, and not all of them are on in a default build. What it cannot tell you is which one you want; these are structural costs, not throughput guarantees:

Backend	Reach	What it buys	What it costs
In memory	Threads in one process	Nothing to configure, nothing left behind, and it is the reference the others are calibrated against	The study dies with the process
Journal file	Processes on a shared filesystem	Resume is replay; no daemon; immutable JSONL records; crash tolerance by construction; designed for NFS with a mount-specific live test	Reading replays the log; broad summaries still materialize history, and there are no indexed queries; an untested NFS export is not certified
SQLite	Many workers on one host	Indexed targeted queries and real SQL for tooling	Broad summaries still materialize required history; it does not work over NFS, and the bundled engine costs a C compile
Remote	Many machines, no shared filesystem	Wraps any other backend and serves it over TCP	A round trip per operation, and no authentication and no encryption — it binds to localhost unless you say otherwise

Journal v6 writes one immutable extent plus a bounded head per append, independent of history length. Unpublished tails never replay. Writable v1–v5 journals migrate once; stop all older handles before upgrading. Keep the <journal>.d/ data directory with the canonical head when copying or deleting a study. Observers (StudyReader::refresh, atune top) rehydrate the full snapshot on every refresh. The atune_core::storage::journal module documentation owns the lease cadence; atune_core::telemetry::Metric documents the journal metrics.

Replay trusts what the journal committed. Opening a journal, or catching up on what other workers appended, applies each committed record through the same step a live write takes after its own checks have passed, and does not repeat those checks. It still refuses a record it cannot apply consistently, such as a gap in the record sequence, an extent whose checksum disagrees, an unknown study or trial, or a state change the trial's current state does not allow, but a well-formed record that a live write would have rejected is applied as written. A hand-edited or corrupt journal can therefore carry operations no live write accepts: a parameter recorded under a distribution that conflicts with the one already recorded for it, or outside its own distribution, and a parameter, report, error message or heartbeat recorded against a finished trial or one that the lifecycle protocol manages. Checking every record again would cost every open, so atune does not.

Naming one

The Rust facade and CLI name each backend with a single string. Python's create_study(storage=…) and load_study(…) accept the in-memory, journal and SQLite forms below; the Python wheel does not enable the facade's remote feature, so atune:// is not a Python locator:

Spec	Backend
:memory:	In memory. This is the default when you say nothing.
study.atj	A journal file — the extension is what selects it
study.db, study.sqlite, study.sqlite3	SQLite
atune://host:port	A remote server (Rust and CLI only)

Naming a backend your build does not have is a clear error saying which cargo feature is missing, not a mysterious failure — and never a silent fallback.

Resuming a study

Because the study is in storage, stopping is not losing. A later handle opens the same file, loads the study by id, and continues: the trial numbering carries on where it left off, the seed and the space come back out of storage, and whatever state the sampler or scheduler persisted is restored. Resume and scale shows the code and checks the resumed study against an uninterrupted one, bit for bit.

Several workers, one study

In one process, threads share a handle and are serialised inside it.

Across processes on a shared filesystem, the journal backend serialises writers with a renewable fenced directory lease, not flock, because POSIX record locks are unreliable over NFS. Readers never take the lease. A claim contains a nonce and renewable epoch; a contender may reclaim it only after the same complete claim stays unchanged for one grace period on the contender's local monotonic clock. A generation-bound append ticket prevents a late old writer from publishing after takeover. Pre-lease lock files are diagnosed but never auto-broken; coordinated cleanup is required before mixing old and new writers. Likewise, a worker killed while breaking a stale lease can leave an incomplete revoke marker that stops reclaim; a lock timeout reached after the lease went stale names the file to remove. The real-NFS ignored test is the evidence boundary for a mount's exclusive-create and atomic-rename behavior.

Across machines without a shared filesystem, the remote backend is the answer, and SQLite is not: its write-ahead log is a shared-memory protocol and does not work over NFS. That is why both file backends exist. The server side of it is atune serve, which takes a spec of its own — so what the workers talk to and what actually holds the study stay two separate decisions.

Remote telemetry separates client-side serialization from service time. The atune_core::metrics target emits remote_queue_time as whole microseconds waiting for that handle's socket mutex and remote_request_latency as whole microseconds after acquisition, including transport, backend work, and any lifecycle reconnect/replay. Their shared operation value joins the two events; it is not a metric label. Lifecycle traffic uses a separate lazy control socket, so a blocked data-page read cannot hold its channel, although calls within each channel still serialize. remote_page_restart counts the one bounded retry after an immutable page session expires or is evicted. remote_replay_eviction counts completed lifecycle results removed by the entry or byte cap: eviction shortens replay availability and never authorizes automatic re-execution. These events expose a workload's shape; atune claims no remote latency or throughput threshold without controlled measurement.

Whichever you pick, running trials write heartbeats. Failover requires the same owner/epoch/heartbeat observation to remain unchanged for one sweeper-local grace period, then atomically records one failed outcome and durable replacement intent when requeue is enabled. A renewal racing that commit leaves either the live owner intact or exactly one complete stale failure. A sweeper that holds an exclusive lock every worker of the study also takes may declare it (FailoverPolicy::with_exclusive_owner): no other live owner can exist, so one observation suffices and the grace period is skipped. A worker treats its own lease as lost only when renewal reports LeaseLost or keeps failing for 30 s after the last success; a transient renewal error is retried. A lease lost the second way fails the trial's intermediate writes promptly, but its pause or terminal commit is still attempted: the fenced terminal write decides who owns the trial, so finished work survives a storage outage that no other worker exploited.

Built-in backends read a StudySnapshot from one backend read epoch; remote storage may page that immutable result. A custom backend using the default read_snapshot is marked BestEffort, because its trial, state, and outbox reads can observe concurrent writes. The default reads outbox records in finite pages, falling back to list_outbox if paging is unsupported or a record exceeds the page budget; a policy-discovery retry is bounded to one additional pass. StudyReader::user_attrs() is a separate read and can observe different metadata from the snapshot.

Writing your own backend

The storage seam has a written contract, backed by a conformance suite that ships behind a cargo feature. Running it against your backend produces one test per invariant, so a failure names the invariant you broke rather than saying that the suite failed. It covers numbering under real thread contention, the transition matrix, immutability, the parameter-write gate, report ordering, state blobs and the synchronisation cursor.

All four backends in this repository are themselves customers of it, and one of them runs the whole suite over TCP. Extending: storage walks through it.

Where to go next

If you want to	Go to
Keep a study across runs and machines	Resume and scale
Read a stored study	Inspect a study
Look a backend up by type, with its feature gate	Storage catalogue
Know what is written into a journal file, record by record	Journal format
Open a study in Optuna's dashboard	Optuna interop
Implement the trait yourself	Extending: storage
Understand what resuming preserves, and why	Determinism