Module exec 

The subprocess executor: run any program, in any language, as a trial.

An InProcess objective is a Rust closure the study loop calls on a thread. A Subprocess objective is a child process the study loop launches, feeds a trial’s sampled parameters, and reads a result back from (tune any program). It is the any-language path — the program can be a shell script, a Python trainer, a compiled binary — and it is the path oniro drives today, because a process-global RNG cannot share an address space with other trials.

The executor sits above ask/tell: for each trial it asks the study for a context, suggests the declared parameters through it (so trial n still gets the sampler’s decision for n, and nothing else), spawns the program with those values, reads the objective from the child’s output, and tells the result back. Because that is all it does, it is an ordinary Objective and rides the existing loop — parallelism, retries, budgets, panic isolation and resume are inherited unchanged, not reimplemented (see Determinism and resume below).

This whole module lives behind the system feature: spawning a child is operating-system code, absent on wasm32-unknown-unknown. It is a reusable atune_core piece rather than a CLI-private helper, because oniro drives the same executor.

use atune_core::exec::{CommandTemplate, Subprocess};
use atune_core::space::dsl;
use atune_core::study::{Budget, Study, StudyConfig};
use std::time::Duration;

// A search space, written in the string DSL.
let schema = dsl::parse_schema([
    ("lr", "loguniform(1e-4, 1e-1)"),
    ("opt", "choice(adam, sgd)"),
])?;

// `./train.sh --lr={lr} --opt {opt}` — argv-level substitution, no shell.
let template = CommandTemplate::new("./train.sh", ["--lr={lr}", "--opt", "{opt}"])?;
let executor = Subprocess::new(schema.clone(), template)?.timeout(Duration::from_secs(600));

let study = Study::builder()
    .parallelism(8) // eight concurrent child processes
    .budget(Budget::trials(200))
    .create(StudyConfig::new("shell-tune").with_seed(1).with_space(schema))?;
executor.run(&study)?;

The command template and the environment handshake

Both channels are supported and may be used together (tune any program).

• Template. CommandTemplate holds a program and argument templates in which {name} is replaced by the rendered value of the parameter name — --lr={lr} or a bare {lr}. Substitution is argv-level: the value becomes one element of the child’s argument vector, or part of one, and is never re-parsed. See the trust model below.
• Environment handshake. Every live-trial child is given ENV_TRIAL (the trial id), ENV_TRIAL_WORKER and ENV_TRIAL_EPOCH (the exact fenced owner operation), ENV_TRIAL_NUMBER (the per-study number), and ENV_SEED (a per-trial objective seed), plus ENV_STUDY. The complete identity is the handshake a scripted worker carries into a nested atune ask/tell; a numeric ATUNE_TRIAL alone is insufficient. Held-out reevaluation is deliberately read-only and omits the three mutation-authority variables while retaining number and seed. With export_params each sampled parameter is also exported as <prefix><name> (default prefix DEFAULT_PARAM_PREFIX), so a program that prefers to read its configuration from the environment can, without an atune ask round trip. A forked or resumed trial is additionally given ENV_CHECKPOINT, the checkpoint reference to restore from (e.g. oniro’s run_training_resume run-dir) — a reference hand-off, never bytes.

The result protocol

The child reports its objective two ways at once, and both are total — neither can make the executor panic:

1. stdout carries the value on its last non-empty line, as a bare number (0.37) or a JSON object with a value or values field ({"values": [0.37, 12.0]} for a multi-objective study).

2. exit status decides the trial’s fate:

   Exit code	Trial state
   0	Complete, with the value parsed from stdout
   PRUNE_EXIT_CODE (42 by default, configurable)	Pruned
   any other non-zero, or a fatal signal	Failed, with stderr kept in the record

   A 0 exit whose stdout does not carry a readable objective is a clean Failed — a program that succeeded but printed garbage is a bug in the program, recorded as a failed trial, never a crash in the tuner.

A completed child may also carry a checkpoint reference on that same JSON line — {"value": 0.37, "checkpoint": "runs/ppo/3", "step": 30000} — which atune records (a reference, never bytes, 03 §5) so a later Fork or resume can hand it back through ENV_CHECKPOINT.

A timeout kills a child that overruns its deadline and records the trial Failed.

Trust and security model

The executor spawns a user-chosen program and substitutes sampled values into its argument vector and environment. It does this without a shell: std::process::Command receives an explicit argv, so there is no /bin/sh -c, no word-splitting, no glob expansion, and no shell-metacharacter injection surface. A sampled value of ; rm -rf / is one literal argument string, not a command.

The trust boundary is therefore the same one the user already accepted when they wrote the search space and named the program: the values substituted are draws from the user’s own declared distributions — a float, an integer, a categorical label, a boolean — not attacker-controlled input from a network. The executor does not sandbox the child; a subprocess trial runs with the same privileges, environment and filesystem access as the process that launched it. Environment variables are set on top of the inherited environment; parameter exports use a non-empty prefix by default precisely so that a parameter named, say, PATH cannot clobber a meaningful variable.

Determinism and resume

The executor adds no randomness. The parameters a child receives for trial number n are exactly what the sampler decided for n, obtained by suggesting each declared parameter through the trial’s context — the same priority chain every objective walks. The child’s own randomness comes from ENV_SEED, which is the trial’s replicate_seed — outside a multi-seed fan that is seed_for(study_seed, number, Stream::Objective), a pure function of the trial number, so a trial reproduces regardless of which worker ran it; under a SeedProtocol fan it is replicate j’s seed, so the child of replicate j gets the common-random-numbers seed.

Because a subprocess run is just Study::optimize_with over a Subprocess objective, a study run this way is resumable exactly like any other: point a fresh executor at a study loaded from a journal and it continues the numbering, re-derives the same seeds, and re-runs only what is left to run.

Structs

CommandTemplate

A program plus its argument templates: the recipe for one child process.

Subprocess

Runs each trial as a child process, feeding it sampled parameters.

SubprocessError

The error recorded for a trial whose child process failed.

Enums

CappedRun

The captured result of run_capped: the child exited, or it overran its deadline and was killed.

Constants

DEFAULT_PARAM_PREFIX

The default prefix for exported parameter environment variables.

ENV_CHECKPOINT

The environment variable carrying a checkpoint reference for the child to resume from.

ENV_SEED

The environment variable carrying the child’s per-trial objective seed.

ENV_STUDY

The environment variable naming the child’s study.

ENV_TRIAL

The environment variable naming the child’s trial (its storage-global id).

ENV_TRIAL_EPOCH

The environment variable carrying the child’s lifecycle ownership epoch.

ENV_TRIAL_NUMBER

The environment variable naming the child’s per-study trial number.

ENV_TRIAL_WORKER

The environment variable carrying the child’s lifecycle owner id.

MAX_OUTPUT_CAPTURE_LIMIT

The largest capture_limit an executor accepts.

PRUNE_EXIT_CODE

The exit code a child uses to say “prune this trial”, by default.

Functions

run_capped

Runs command to completion under an optional wall-clock timeout, capturing stdout and stderr, and never hanging past a bounded margin of the child’s exit no matter what it spawned.

run_capped_in_parent_group

Runs command under an already trusted caller-owned Unix process group.