feat(slot_state): park-epoch in the slot word; consuming, epoch-matched unparks

Word repacks to (gen << 32) | (epoch << 8) | state. begin_wait opens a wait
identity; every successful wake consumes it, so at most one wake lands per
wait by construction. unpark grows an Option<epoch> match; RuntimeInner gains
unpark_at. Claim/yield/park transitions become epoch-preserving CAS loops.

Loom: all four prior theorems re-proved on the new word, plus
consumed_epoch_unpark_never_lands — a late waker stamped with a consumed
epoch can neither enqueue nor notify, in every interleaving (the property
select's loser arms and every one-shot park site lean on).

No callers stamped yet; behaviour identical. Two transient dead_code
warnings (begin_wait, unpark_at) are consumed by the next commit.
This commit is contained in:
smarm
2026-06-10 07:07:44 +00:00
parent aa295582c4
commit 4913835c02
2 changed files with 267 additions and 79 deletions
+29 -9
View File
@@ -28,7 +28,7 @@
//! # The per-slot state machine
//!
//! Scheduling state lives in one atomic word per slot packing
//! `(generation, state)`, where state is one of:
//! `(generation, park-epoch, state)`, where state is one of:
//!
//! ```text
//! Vacant ─spawn→ Queued ─pop→ Running ─yield→ Queued
@@ -44,6 +44,14 @@
//!
//! - The generation check is **atomic with the transition** — a stale `Pid`
//! can never act on a recycled slot (no ABA, no spurious unparks).
//! - The park-epoch (middle 24 bits) is the actor's *wait identity*: opened
//! by `begin_wait` before any registration, consumed (bumped) by every
//! successful wake. Registration-based wakers carry `(pid, epoch)` and use
//! `unpark_at`, so a waker holding a registration from an already-woken
//! wait — a `select` loser arm, a satisfied wait's timer — fails the epoch
//! check and no-ops instead of faulting a later one-shot park. The only
//! wildcard wake is `request_stop`, which is terminal. Full rules in
//! slot_state.rs.
//! - `RunningNotified` replaces the old `pending_unpark` bool: an unpark that
//! races the prep-to-park window is a *state*, resolved by the scheduler's
//! park-return CAS, not a flag read under a lock. This also closes a latent
@@ -509,12 +517,11 @@ impl RuntimeInner {
// move the word off Queued until this very entry is popped — so the
// word must read EXACTLY (gen, Queued) here. This is the at-most-once-
// enqueued invariant the bounded rings' capacity proof leans on.
debug_assert_eq!(
self.slot_at(pid).map(|s| s.word.load()),
Some(crate::slot_state::pack(
pid.generation(),
crate::slot_state::ST_QUEUED
)),
debug_assert!(
self.slot_at(pid).map(|s| s.word.load()).is_some_and(|w| {
crate::slot_state::word_gen(w) == pid.generation()
&& crate::slot_state::word_state(w) == crate::slot_state::ST_QUEUED
}),
"enqueue of a pid not in (gen, Queued)"
);
self.run_queue.push(pid);
@@ -522,10 +529,23 @@ impl RuntimeInner {
}
/// Make `pid` runnable if it is parked; coalesce or defer otherwise.
/// The runtime-internal core of `scheduler::unpark`.
/// The runtime-internal core of `scheduler::unpark`. WILDCARD wake:
/// consumes the epoch but does not check it — reserved for terminal
/// wakes (`request_stop`); see slot_state.rs.
pub(crate) fn unpark(&self, pid: Pid) {
self.unpark_inner(pid, None);
}
/// Epoch-matched wake: lands only if `pid`'s current wait is still the
/// one the waker registered for. The form every registration-based waker
/// (channel senders, mutex grants, wait-timers, …) must use.
pub(crate) fn unpark_at(&self, pid: Pid, epoch: u32) {
self.unpark_inner(pid, Some(epoch));
}
fn unpark_inner(&self, pid: Pid, want: Option<u32>) {
if let Some(slot) = self.slot_at(pid) {
match slot.word.unpark(pid.generation()) {
match slot.word.unpark(pid.generation(), want) {
Unpark::Enqueue => {
crate::te!(crate::trace::Event::UnparkDirect(pid));
self.enqueue(pid);