Module online 

In-loop online tuning — a real-time hyperparameter tuner that lives inside one training run.

This is the headline RL feature only a native tuner can offer: the ask/tell Study loop tunes across runs, but an OnlineTuner adapts hyperparameters during a single run, from the reward signal it is fed each round. Its decision cost is bounded and tiny — the suggest/observe hot path is O(clusters), allocation-free and CPU-pure (no linalg, no GPU, D4) — so it is safe to call on oniro’s real-time side (async SAC at 500 Hz with hard deadlines).

use atune_core::online::{Mabc, OnlineTuner};

let mut tuner = OnlineTuner::builder()
    .cluster("lr", &[1e-4, 3e-4, 1e-3])
    .cluster("ent_coef", &[0.0, 0.01, 0.02])
    .policy(Mabc::new_with_ucb(32)) // ULTHO-style clustered UCB, window = 32
    .build()?;

for _round in 0..3 {
    let (lr, ent) = {
        let hp = tuner.suggest(); // zero-alloc, O(clusters)
        (hp.get("lr").unwrap(), hp.get("ent_coef").unwrap())
    };
    // ...train one round with `lr` / `ent`, producing a return...
    let round_return = 1.0 - (lr - 3e-4).abs() - ent; // stand-in
    tuner.observe(round_return); // sliding-window credit
}

The two policies

• Mabc — the ULTHO-style clustered sliding-window UCB bandit over discrete candidate values per cluster. The primary policy; the exact UCB rule is pinned and golden-tested.
• PbtLite — a deterministic scalar-perturbation policy for continuous knobs: periodic ×up/×down explore steps, no forking, no RNG.

Determinism and portability

The built-in policies are deterministic by construction. The callback seam also receives a pinned per-round [rand_chacha::ChaCha8Rng] derived from the configured seed, so callback randomness replays from the seed and round ordinal (the determinism contract). The module is atune_core, so it inherits the wasm floor and the ground rules: no faer, no std::time, no threads, and no allocation on the hot path.

Still a replayable study

The hot path touches no storage, but the tuner can be snapshotted into a crate::storage::StateBlob at interval boundaries — never on the hot path — so an online run remains a replayable, resumable study (snapshot/restore, is_flush_boundary).

The callback seam

drive_round provides a consumer-agnostic per-round callback shape: it supplies the suggested Choice, passes a deterministic round-specific [rand_chacha::ChaCha8Rng], and observes the return value produced by the callback. Callers can use it to run any round-level work; it does not define a consumer integration. The honest ablation lives in atune_bench (online_oracle).

Structs

Choice

The set of hyperparameter choices OnlineTuner::suggest produced this round — one value per cluster, borrowed from the tuner (no allocation).

Mabc

The ULTHO-style clustered UCB policy: a sliding-window UCB1 bandit over each cluster’s discrete candidate values.

OnlineTuner

A real-time, in-loop hyperparameter tuner.

OnlineTunerBuilder

A builder for an OnlineTuner.

PbtLite

The deterministic PBT-lite scalar-perturbation policy for continuous knobs.

Enums

Policy

Which in-loop policy an crate::online::OnlineTuner runs. Constructed from a Mabc or a PbtLite via crate::online::OnlineTunerBuilder::policy (both impl Into<Policy>). This enum may gain variants; downstream matches must handle unknown variants.

Constants

DEFAULT_EXPLORATION

The default UCB exploration constant c (the classic UCB1 value √2).

DEFAULT_PERTURB_DOWN

The default PBT-lite explore down factor.

DEFAULT_PERTURB_UP

The default PBT-lite explore up factor.

DEFAULT_WINDOW

The default sliding window for Mabc::new_with_ucb, in rounds.

MAX_WINDOW

The largest Mabc window accepted — bounds the ring-buffer allocation so a user-supplied window can never size an unbounded allocation.

ONLINE_BLOB_KIND

The kind tag of the persisted crate::online::OnlineTuner snapshot (crate::storage::StateBlob).

ONLINE_BLOB_VERSION

The schema version of the ONLINE_BLOB_KIND payload.