Skip to main content

Module sampler

Module sampler 

Expand description

The samplers the benchmark surface exposes behind a flag.

Two tiers, deliberately kept apart:

  • The gate kinds (SamplerKind::ALL) — the uniform Random baseline, the quasi-random Sobol Qmc, and the learning Tpe. This array is the M2.3 matrix: the quality oracle iterates it and pins a golden curve against it, so it is a contract rather than a convenience. Adding a kind to it silently changes what the gate measures.
  • The frontier kinds (SamplerKind::FRONTIER) — the M6 facade samplers Gp (GP-EI) and Cmaes, plus the rule-based Auto picker. They exist so atune-solver --sampler can put the frontier work on an external benchmark’s axes (kurobako, next to Optuna’s own solvers), and they stay out of the in-tree gate on purpose: each already has a dedicated oracle with a differential and a regression bite (tests/gp_oracle.rs, tests/cmaes_oracle.rs), which pins far more than a curve in the M2.3 matrix would, and Auto is not a sampler at all — it is a pick, so it has no behaviour of its own to pin.

Every kind is built with its default configuration; the point of the gate is to guard those defaults, so the harness deliberately exposes no knobs.

§Why CARBS is not a kind

Carbs is the one M6 sampler deliberately left out, and the omission is load-bearing rather than an oversight. CARBS is cost-aware: it requires a two-objective study whose second declared objective is a Minimize cost dimension, and it refuses any other shape in infer_relative_space instead of silently optimizing the wrong thing. A kurobako problem reports a single objective value and has no channel for a cost — the adapter does not even advertise the MULTI_OBJECTIVE capability — so a carbs arm would return an error for every single ask. Adding one needs a cost-carrying problem source, not a new match arm here.

§What auto can resolve to (including samplers with no kind of their own)

Auto is a pick, not a sampler, and its rule ladder (atune::auto_sampler()) can land on samplers this enum does not name. Two matter to anyone reading an atune-auto benchmark result:

  • Grid (rule 3) for a finite space of at most 256 points that the declared budget can enumerate — and Grid exhausts. The kurobako adapter answers past that point with uniform Random rather than failing the run; see the kurobako module docs, A search space that runs out, for what the substitution means for the resulting number.
  • Nsga2 (rule 1) for a multi-objective study. Unreachable through the kurobako adapter, whose sessions are single-objective by construction (the solver does not advertise MULTI_OBJECTIVE, so kurobako refuses to start such a study), but reachable through build_for with a multi-direction argument.

Rules 4–7 land on Qmc, Tpe, GpEi or Cmaes, each of which does have its own kind here.

Enums§

SamplerKind
Which built-in sampler to run.