Function hssp 

pub fn hssp(
    points: &[Vec<f64>],
    subset_size: usize,
    reference: &[f64],
    directions: &[Direction],
) -> Result<Vec<usize>>

Hypervolume subset selection: pick subset_size of points whose joint hypervolume against reference is as large as possible, greedily.

The exact problem is NP-hard; the greedy answer — repeatedly add the point with the largest hypervolume contribution to those already chosen — is a 1 − 1/e approximation because hypervolume is monotone submodular (Guerreiro, Fonseca, Paquete 2016). Its selection and tie rules follow Optuna’s _solve_hssp (optuna/_hypervolume/hssp.py), the tie-breaker MOTPE uses to split the last non-domination rank that does not fit into its “below” set. Near the exponent limits, scaled volumes retain contribution ordering beyond f64’s range while keeping f64 precision.

Returns indices into points, sorted ascending.

What is selected, in order

1. If subset_size == points.len(), every index.
2. Only contributing points take part in the greedy step: those with one coordinate per direction, finite, that strictly beat reference in every objective (the hypervolume skipping rules). Exact duplicates among them are collapsed onto their first occurrence, as Optuna does with np.unique(..., return_index=True).
3. If there are at least subset_size unique contributing points, the greedy selection runs on them. Otherwise all of them are taken and the rest is filled — each group in ascending index order — first with the duplicates collapsed in step 2 (Optuna’s unique-loss handling), then with the non-contributing points, which add no volume whichever are chosen.

Greedy details

• Two objectives, mutually non-dominated points (the MOTPE case: one non-domination rank): Optuna’s exact O(k·N) 2-D update, which tracks each candidate’s contribution as one rectangle.
• Otherwise: Optuna’s lazy greedy. A candidate’s previous contribution and its inclusion–exclusion bound against the latest selected point are both upper bounds on its new contribution (submodularity), so candidates are re-evaluated from the largest bound down and skipped once the bound falls below the best exact contribution found. Re-evaluation adds the candidate to the selected set and differences the hypervolume (up to three objectives), or subtracts the hypervolume of the selected set limited to the candidate from the candidate’s box (above three, with hypervolume_wfg’s WFG).

Determinism

No randomness, and ties are broken stably: among equal contributions the lexicographically smallest minimised vector wins (Optuna’s argmax over its np.unique-sorted array), and equal vectors resolve to the lowest index. The input order affects the result only through those index ties.

Errors

Error::InvalidSpace if directions is empty, if reference does not have one entry per direction, if reference carries a non-finite value, or if subset_size exceeds points.len().

use atune_core::pareto::hssp;
use atune_core::study::Direction::Minimize;

let front = vec![vec![1.0, 3.0], vec![2.0, 2.0], vec![3.0, 1.0]];
// (2, 2) covers 4 of the 6 units of area on its own; after it, the two
// corners add one unit each, and the tie goes to the lexicographically
// smaller (1, 3).
assert_eq!(hssp(&front, 1, &[4.0, 4.0], &[Minimize, Minimize])?, vec![1]);
assert_eq!(hssp(&front, 2, &[4.0, 4.0], &[Minimize, Minimize])?, vec![0, 1]);