Skip to content

Optimise several objectives, under constraints

Goal: tune for two or more things that genuinely conflict, and get the set of sensible trade-offs rather than one arbitrary compromise.

Accuracy against latency. Quality against cost. Reward against energy. When objectives conflict there is no single best configuration, and collapsing them into one number — a weighted sum — decides the trade-off before you have seen it. A multi-objective study defers that decision to you by returning the Pareto front: every configuration that nothing else beats on all objectives at once.

Declare a vector objective

let study = Study::builder()
    .parallelism(THREADS)
    .budget(Budget::new_with_trials(TRIALS))
    .sampler(Arc::new(Nsga2::try_new_with_population_size(POPULATION)?))
    .create(
        StudyConfig::new(STUDY)
            .with_directions([Direction::Minimize, Direction::Minimize])?
            .with_metric_names(METRICS)?
            .with_seed(SEED),
    )?;

study.optimize(evaluate)?;

// The answer: every configuration nothing else beats on both objectives.
let mut front = study.pareto_front()?;
front.sort_by_key(|trial| trial.number);

study = atune.create_study(
    direction=["minimize", "minimize"],
    metric_names=METRICS,
    sampler=atune.samplers.Nsga2(population_size=POPULATION),
    seed=SEED,
    name=STUDY,
)
study.optimize(objective, n_trials=TRIALS, n_jobs=JOBS)

# The answer: every configuration nothing else beats on both objectives.
front = sorted(study.pareto_front(), key=lambda trial: trial.number)

The direction is per objective, so Minimize error alongside Minimize cost, or maximise one and minimise the other. Nsga2 does the ranking — non-dominated sorting for who survives, crowding distance for keeping the front spread out rather than clustered.

Set the population size deliberately. Nsga2::try_new_with_population_size(16) against a 48-trial budget gives three generations; the default of 50 would mean the population never fills, so nothing is ever bred and the study degenerates into random search wearing a genetic algorithm's name.

Read the front, not the best

study.best_trial() returns None for a multi-objective study, on purpose: there is no single best, and returning an arbitrary member of the front would be a lie the type system can prevent. Use the front instead: pareto_front() in Rust, the best_trials property in Python.

The front is what you present. Its size is also the first diagnostic: a front holding every trial in the study means no configuration is simply worse than another, so the study cannot tell waste from trade-off.

Make sure your objectives conflict

This is the trap, and it is easy to walk into with a plausible-looking objective.

If one objective is a strictly monotone function of the other and their directions pull against each other, no configuration can dominate any other — every trial is non-dominated and the front is the whole study. Check that the front is a fraction of the trials before trusting the trade-offs.

Constraints

A constraint is different from an objective: you do not want to trade it off, you want it respected. The choice depends on whether a violation is knowable up front.

If it is knowable from the parameters alone, express it in the search space so infeasible configurations are never proposed. A width that must be a multiple of eight is a stepped distribution, not a penalty.

If it is only knowable after evaluating, a Rust objective records it in the canonical constraint side-channel: call TrialCtx::record_constraints(&[...]). Values ≤ 0 are satisfied and values > 0 are violated. These values are typed trial state, not another objective column, so constrained dominance puts declared feasible trials first, ranks declared infeasible trials by total violation, and does not treat an undeclared trial as proof of feasibility. A multi-seed fan records one vector per replicate and carries the componentwise worst vector, so feasibility must hold for every seed.

Which samplers act on those values is a separate question. Nsga2 always ranks feasibility-first. A single-objective study using Tpe::new().with_constraints(true) builds TPE's good set from feasible trials while any exist, and from the least violating trials otherwise, so the search moves toward the best feasible region rather than the best region overall. Every other sampler, and Tpe by default, models the objective alone; best_trial() still ranks feasibility-first.

In Python, Trial.set_constraint(key, value) records named, finite float-castable values through that same canonical seam. Repeated keys warn and keep the first value; every declared vector in a study must share one key set. Decimal names sort numerically first (lexical ties), then other names lexically, matching Optuna import. Trial.constraints is a dict copy; FrozenTrial.constraints remains the canonical vector (None means no declaration).

Use atune.samplers.Tpe(constraints=True) to enable constrained TPE. NSGA-II always uses constraints; declare create_study(search_space=...) with atune distribution description dicts to activate its population proposals. Without a declared space it keeps the independent fallback. The tested migration example exercises both samplers, named constraints, and multi-objective directions.

Resist the third option — a large penalty added to another objective. It makes the objective discontinuous exactly where the sampler most needs to model it, and it turns "infeasible" into "very slightly worse than feasible" at whatever scale you picked for the penalty.

Python and Rust differ here

Study::hypervolume and nadir_point exist on the Rust Study and are not exposed in the Python binding. The standard scalar summary of a multi-objective run is therefore available in Rust only; from Python, present the front itself.

Next