Skip to main content

Module training

Module training 

Expand description

The simulated training run a scheduled trial reports from.

Pruners and population schedulers decide on intermediate reports, which a closed-form function does not have. With a scheduler, or with DEHB, a trial is therefore a short training run at its point (x, y): its loss starts at the problem’s ceiling and its gap to f(x, y) shrinks quadratically to zero over STEPS steps. Each report multiplies that gap by 1 + NOISE·u, with u ∈ [−1, 1) drawn from the trial’s seed and the step, so curves wiggle like real ones without ever reporting below f, and a full-length run ends exactly at f.

Every function here is pure and uses only integer hashing and basic float arithmetic, so the browser build reports the same curves as the host.

Constants§

NOISE
The relative noise on a report’s gap to f(x, y).
STEPS
Steps in one full training run; also the top of the schedulers’ ladder.

Functions§

loss
The training loss at step of a segment that started at start with loss start_loss and converges to value at horizon. Past the horizon it stays at value.
reported
The value a trial reports at step: the training loss with deterministic noise on its gap to value.
unit_noise
A value in [−1, 1) from seed and step (SplitMix64 finalizer).