Module seed_protocol 

The multi-seed protocol: trials as fans of replicates (RL headline #1).

The ICML-2023 finding — reinforcement-learning configurations overfit their tuning seeds, sometimes catastrophically — is a type here, not a footnote (use a multi-seed evaluation protocol). A SeedProtocol turns one trial into a fan of replicates: the same configuration evaluated under k seeds. The per-seed values are stored as sub-results and never lost; the sampler sees their Aggregate as the trial’s objective.

use atune_core::seed_protocol::{Aggregate, Pairing, SeedProtocol};

// Three paired (common-random-numbers) tune seeds, aggregated with the
// interquartile mean, and five disjoint test seeds for the final stage.
let protocol = SeedProtocol::new()
    .with_tune_seeds(3)
    .with_test_seeds(5)
    .with_pairing(Pairing::Paired)
    .with_aggregate(Aggregate::Iqm);

assert!(protocol.fans());
assert_eq!(protocol.effective_tune_seeds(), 3);
// The default is single-seed: no fan, nothing changes for a non-RL study.
assert!(!SeedProtocol::new().fans());

The three moving parts

• The fan width (SeedProtocol::tune_seeds) — k replicates per trial. The default is 1 (OFF): a single-seed study behaves exactly as it did before this protocol existed. k >= 2 turns on the fan.
• The pairing (Pairing) — Paired is common random numbers: replicate j draws the same seed in every trial, so two configurations are compared under identical randomness (variance reduction, offered by nobody mainstream). Independent folds the trial number in too, so every replicate of every trial is a fresh draw.
• The aggregate (Aggregate) — how the k per-seed values collapse into the one number the sampler ranks: Mean, Median, Iqm (the rliable lesson: robust to outlier seeds) or MeanMinusStd (risk-averse).

The seed formulas (the load-bearing determinism detail)

Every replicate seed is a pure function of the study seed and the pairing, built from the frozen seed_for mixing function so it inherits the determinism contract:

Pairing	Tune seed of replicate j	Test seed of replicate j
Paired	seed_for(study_seed, PAIRED_TRIAL_SLOT, Replicate(j))	seed_for(study_seed, PAIRED_TRIAL_SLOT, Replicate(TEST_REPLICATE_OFFSET + j))
Independent	seed_for(study_seed, trial_number, Replicate(j))	seed_for(study_seed, trial_number, Replicate(TEST_REPLICATE_OFFSET + j))

The paired row does not contain the trial number — that is the whole point: replicate j of trial i and of trial i + 1 draw the identical seed, which is common random numbers. The tune and test streams never collide because the test replicate index is shifted by TEST_REPLICATE_OFFSET.

Structs

ReplicateFan

The whole replicate fan of one trial: its tune replicates and, once the re-evaluation stage has run, its test replicates.

ReplicateResult

One replicate’s result within a fan.

SeedProtocol

The seed protocol attached to a study: the fan width, the pairing and the aggregate.

Enums

Aggregate

How the per-seed values of a fan collapse into one sampler-visible number.

Pairing

How a replicate’s seed depends on the trial it belongs to.

Constants

FAN_BLOB_KIND

The kind tag of the per-trial replicate-fan state blob.

FAN_BLOB_VERSION

The schema version of the FAN_BLOB_KIND payload (ReplicateFan).

MAX_FAN_WIDTH

The largest fan width, tune or test, a study configuration may declare.

PAIRED_TRIAL_SLOT

The fixed trial-number slot paired (Pairing::Paired) seeds are derived from, in place of the real trial number.

TEST_REPLICATE_OFFSET

The offset added to a test replicate’s index so the test seed stream is disjoint from the tune stream.

Functions

aggregate_columns

Aggregates a fan of multi-objective replicate values, one aggregate per objective dimension.

fan_scope

The independently keyed state scope for a trial’s replicate fan.

overfit_gap

The signed tune-vs-test gap, positive when a configuration looked better on tune seeds than it held up on test seeds — the ICML-2023 overfitting the protocol guards against.