feat(slot_state): park-epoch in the slot word; consuming, epoch-matched unparks
Word repacks to (gen << 32) | (epoch << 8) | state. begin_wait opens a wait identity; every successful wake consumes it, so at most one wake lands per wait by construction. unpark grows an Option<epoch> match; RuntimeInner gains unpark_at. Claim/yield/park transitions become epoch-preserving CAS loops. Loom: all four prior theorems re-proved on the new word, plus consumed_epoch_unpark_never_lands — a late waker stamped with a consumed epoch can neither enqueue nor notify, in every interleaving (the property select's loser arms and every one-shot park site lean on). No callers stamped yet; behaviour identical. Two transient dead_code warnings (begin_wait, unpark_at) are consumed by the next commit.
This commit is contained in:
+29
-9
@@ -28,7 +28,7 @@
|
||||
//! # The per-slot state machine
|
||||
//!
|
||||
//! Scheduling state lives in one atomic word per slot packing
|
||||
//! `(generation, state)`, where state is one of:
|
||||
//! `(generation, park-epoch, state)`, where state is one of:
|
||||
//!
|
||||
//! ```text
|
||||
//! Vacant ─spawn→ Queued ─pop→ Running ─yield→ Queued
|
||||
@@ -44,6 +44,14 @@
|
||||
//!
|
||||
//! - The generation check is **atomic with the transition** — a stale `Pid`
|
||||
//! can never act on a recycled slot (no ABA, no spurious unparks).
|
||||
//! - The park-epoch (middle 24 bits) is the actor's *wait identity*: opened
|
||||
//! by `begin_wait` before any registration, consumed (bumped) by every
|
||||
//! successful wake. Registration-based wakers carry `(pid, epoch)` and use
|
||||
//! `unpark_at`, so a waker holding a registration from an already-woken
|
||||
//! wait — a `select` loser arm, a satisfied wait's timer — fails the epoch
|
||||
//! check and no-ops instead of faulting a later one-shot park. The only
|
||||
//! wildcard wake is `request_stop`, which is terminal. Full rules in
|
||||
//! slot_state.rs.
|
||||
//! - `RunningNotified` replaces the old `pending_unpark` bool: an unpark that
|
||||
//! races the prep-to-park window is a *state*, resolved by the scheduler's
|
||||
//! park-return CAS, not a flag read under a lock. This also closes a latent
|
||||
@@ -509,12 +517,11 @@ impl RuntimeInner {
|
||||
// move the word off Queued until this very entry is popped — so the
|
||||
// word must read EXACTLY (gen, Queued) here. This is the at-most-once-
|
||||
// enqueued invariant the bounded rings' capacity proof leans on.
|
||||
debug_assert_eq!(
|
||||
self.slot_at(pid).map(|s| s.word.load()),
|
||||
Some(crate::slot_state::pack(
|
||||
pid.generation(),
|
||||
crate::slot_state::ST_QUEUED
|
||||
)),
|
||||
debug_assert!(
|
||||
self.slot_at(pid).map(|s| s.word.load()).is_some_and(|w| {
|
||||
crate::slot_state::word_gen(w) == pid.generation()
|
||||
&& crate::slot_state::word_state(w) == crate::slot_state::ST_QUEUED
|
||||
}),
|
||||
"enqueue of a pid not in (gen, Queued)"
|
||||
);
|
||||
self.run_queue.push(pid);
|
||||
@@ -522,10 +529,23 @@ impl RuntimeInner {
|
||||
}
|
||||
|
||||
/// Make `pid` runnable if it is parked; coalesce or defer otherwise.
|
||||
/// The runtime-internal core of `scheduler::unpark`.
|
||||
/// The runtime-internal core of `scheduler::unpark`. WILDCARD wake:
|
||||
/// consumes the epoch but does not check it — reserved for terminal
|
||||
/// wakes (`request_stop`); see slot_state.rs.
|
||||
pub(crate) fn unpark(&self, pid: Pid) {
|
||||
self.unpark_inner(pid, None);
|
||||
}
|
||||
|
||||
/// Epoch-matched wake: lands only if `pid`'s current wait is still the
|
||||
/// one the waker registered for. The form every registration-based waker
|
||||
/// (channel senders, mutex grants, wait-timers, …) must use.
|
||||
pub(crate) fn unpark_at(&self, pid: Pid, epoch: u32) {
|
||||
self.unpark_inner(pid, Some(epoch));
|
||||
}
|
||||
|
||||
fn unpark_inner(&self, pid: Pid, want: Option<u32>) {
|
||||
if let Some(slot) = self.slot_at(pid) {
|
||||
match slot.word.unpark(pid.generation()) {
|
||||
match slot.word.unpark(pid.generation(), want) {
|
||||
Unpark::Enqueue => {
|
||||
crate::te!(crate::trace::Event::UnparkDirect(pid));
|
||||
self.enqueue(pid);
|
||||
|
||||
Reference in New Issue
Block a user