Struct FailoverPolicy
pub struct FailoverPolicy { /* private fields */ }Expand description
Stateful stale-owner policy for repeated fail_over_stale sweeps.
A policy must be reused across sweeps. Staleness is proved by observing the
same complete lease twice, at least grace_millis apart on the sweeper’s
local monotonic clock. Owner and sweeper wall clocks are never compared.
This deliberately adds one grace interval after a sweeper first discovers
an abandoned lease; a clock-skewed healthy owner cannot be stolen merely
because its heartbeat timestamp is far behind another node’s clock.
A zero grace permits fail-over on the next sweep after that first
observation; it deliberately accepts the risk that a live worker’s renewal
is delayed. The system runtime attempts renewal every second, but this is
not a timing guarantee: choose a grace longer than that cadence with margin
for scheduling and backend I/O. A shorter grace can expire live work.
The worker is used in the explanatory error, and requeue controls whether
the failed configuration is created again as a waiting trial.
A sweeper that holds exclusive ownership of the study by some external
means — a lock file, a job scheduler’s single-instance guarantee — can opt
out of the second observation with
with_exclusive_owner: no other
live worker can hold a lease then, so every running trial it observes is
stale and is reclaimed on the first sweep.
use atune_core::id::WorkerId;
use atune_core::storage::FailoverPolicy;
let policy = FailoverPolicy::new(60_000, WorkerId::new(7)).with_requeue(true);
assert_eq!(policy.grace_millis(), 60_000);
assert_eq!(policy.worker(), WorkerId::new(7));
assert!(policy.requeue());Implementations§
§impl FailoverPolicy
impl FailoverPolicy
pub const fn with_exclusive_owner(self, exclusive: bool) -> Self
pub const fn with_exclusive_owner(self, exclusive: bool) -> Self
Declares that the sweeper exclusively owns the study, so one complete lease observation proves a running trial stale and the grace interval is not waited out.
Only set this while holding an exclusive lock that every worker of the
study also takes: the policy trusts the claim and fails over a healthy
owner’s trial if it is false. The commit still compares the observed
lease atomically, so a renewal or terminal write between observation
and commit wins intact. The default (false) keeps the two-observation
rule.
pub const fn exclusive_owner(&self) -> bool
pub const fn exclusive_owner(&self) -> bool
Returns whether the sweeper declared exclusive ownership.
pub fn with_requeue(self, requeue: bool) -> Self
pub fn with_requeue(self, requeue: bool) -> Self
Sets whether successfully failed-over configurations are re-enqueued.
pub const fn grace_millis(&self) -> u64
pub const fn grace_millis(&self) -> u64
Returns the heartbeat grace period in milliseconds.