Module training
Expand description
The simulated training run a scheduled trial reports from.
Pruners and population schedulers decide on intermediate reports, which a
closed-form function does not have. With a scheduler, or with DEHB, a trial
is therefore a short training run at its point (x, y): its loss starts at
the problem’s ceiling and its gap to
f(x, y) shrinks quadratically to zero over STEPS steps. Each report
multiplies that gap by 1 + NOISE·u, with u ∈ [−1, 1) drawn from the
trial’s seed and the step, so curves wiggle like real ones without ever
reporting below f, and a full-length run ends exactly at f.
Every function here is pure and uses only integer hashing and basic float arithmetic, so the browser build reports the same curves as the host.
Constants§
- NOISE
- The relative noise on a report’s gap to
f(x, y). - STEPS
- Steps in one full training run; also the top of the schedulers’ ladder.
Functions§
- loss
- The training loss at
stepof a segment that started atstartwith lossstart_lossand converges tovalueathorizon. Past the horizon it stays atvalue. - reported
- The value a trial reports at
step: the traininglosswith deterministic noise on its gap tovalue. - unit_
noise - A value in
[−1, 1)fromseedandstep(SplitMix64finalizer).