Markk116/smarm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(safety): phase 5 — loom model checking + invariant audit/assertion sweep

Browse Source 
State machine extracted to src/slot_state.rs as a standalone unit (StateWord:
publish_queued / try_claim / yield_return / park_return / unpark / set_done /
reclaim, plus the Status view for cold paths). runtime.rs keeps the protocol
rationale and consumes the mechanism; every transition self-asserts its
precondition per the assert-the-invariants rule.

src/sync_shim.rs: std vs loom indirection (atomics + UnsafeCell with the
with/with_mut access API) for the two loom-modeled modules. Loom models run
the PRODUCTION code, not a replica:
- slot_state: no-lost-wakeup (park vs unpark), two-unparkers-one-enqueue,
  stale-unpark-never-hits-reused-slot (the ABA theorem), unpark-vs-claim.
- run_queue: mpmc exactly-once through a lap wraparound, push/pop race,
  striped two-producer drain.
RUSTFLAGS="--cfg loom" cargo test --lib --release — 7 models, all pass.
RawMutex deliberately not loom-modeled (futexes can't be; textbook mutex3
with stress + unwind coverage).

Assertion sweep (invariants now checked at the point of reliance):
- enqueue debug-asserts the word reads EXACTLY (gen, Queued) — the
  at-most-once-enqueued invariant the ring capacity proof leans on.
- RawMutex enforces the leaf rule mechanically: debug-build thread-local
  held-count, panics at the acquisition that violates it.
- live_actors underflow (double finalize) asserted.
- StateWord transition preconditions asserted (yield/park/done/reclaim/
  publish/claim).

Audit: with_runtime, RawMutex guards, run-queue ops, and trace::record all
gate preemption (and thereby the stop sentinel) for their span; trace was
already self-gating. Sole remaining exception is the channel std MutexGuard
— the documented first post-v0.5 fast-follow.

Review fixes to the branched-in work:
- slot_state::status_for mapped (matching gen, Vacant) to Stale instead of
  unreachable!: Pid::new is public, so a forged/never-issued pid (e.g.
  monitor(Pid::new(5,0)) on a fresh slab) could reach it — "no such actor"
  is the correct total answer; issued pids still can't get there.
- Tightened enqueue's assert from Status::Live to the exact (gen, Queued)
  word its own comment argues for.

Validated: 22 suites in debug (all asserts + leaf counter live) for
rq-mutex/rq-mpmc/rq-striped, release for rq-mutex, smarm-trace build +
stress, loom 7/7.
This commit is contained in:
Claude
2026-06-09 21:27:45 +00:00
parent 6d9f3698d4
commit 039703dbeb
9 changed files with 651 additions and 156 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/runtime.rs
+55 -127
View File
@@ -104,6 +104,7 @@ use crate::monitor::{Down, DownReason, MonitorId};
use crate::pid::Pid;
use crate::preempt::PREEMPTION_ENABLED;
use crate::raw_mutex::RawMutex;
use crate::slot_state::{StateWord, Status, Unpark};
use crate::supervisor::Signal;
use crate::timer::Timers;
use crate::context::{get_actor_sp, set_actor_sp, switch_to_actor};
@@ -304,27 +305,6 @@ impl RuntimeStats {
pub(crate) const ACTOR_STACK_SIZE: usize = 64 * 1024;
// State encodings for the low 8 bits of `Slot::word`.
pub(crate) const ST_VACANT: u64 = 0;
pub(crate) const ST_QUEUED: u64 = 1;
pub(crate) const ST_RUNNING: u64 = 2;
pub(crate) const ST_RUNNING_NOTIFIED: u64 = 3;
pub(crate) const ST_PARKED: u64 = 4;
pub(crate) const ST_DONE: u64 = 5;
#[inline]
pub(crate) const fn pack(gen: u32, st: u64) -> u64 {
((gen as u64) << 32) | st
}
#[inline]
pub(crate) const fn word_gen(w: u64) -> u32 {
(w >> 32) as u32
}
#[inline]
pub(crate) const fn word_state(w: u64) -> u64 {
w & 0xFF
}
pub(crate) type Closure = Box<dyn FnOnce() + Send>;
/// Lifecycle data, mutated only under the slot's cold [`RawMutex`]. Everything
@@ -357,8 +337,9 @@ pub(crate) struct SlotCold {
/// between unrelated actors.
#[repr(align(128))]
pub(crate) struct Slot {
/// `(generation << 32) | state`. See the module docs for the machine.
word: AtomicU64,
/// `(generation << 32) | state` — the state machine, factored into
/// `slot_state.rs` (loom-modeled there; every transition self-asserts).
word: StateWord,
/// Saved stack pointer. Written by the owning scheduler thread before the
/// Release transition out of Running; read after the Acquire transition
/// Queued→Running. Relaxed is sufficient — ordering rides on `word`.
@@ -377,7 +358,7 @@ pub(crate) struct Slot {
impl Slot {
fn vacant() -> Self {
Self {
word: AtomicU64::new(pack(0, ST_VACANT)),
word: StateWord::new(),
sp: AtomicUsize::new(0),
stop_ptr: AtomicPtr::new(std::ptr::null_mut()),
closure: AtomicPtr::new(std::ptr::null_mut()),
@@ -394,16 +375,18 @@ impl Slot {
}
}
#[inline]
pub(crate) fn word(&self) -> u64 {
self.word.load(Ordering::Acquire)
}
/// Current generation (of whatever occupies the slot — pair with a state
/// check or a CAS before acting on it).
/// Current generation (of whatever occupies the slot — pair with a
/// status check or a CAS before acting on it).
#[inline]
pub(crate) fn generation(&self) -> u32 {
word_gen(self.word())
self.word.generation()
}
/// A pid's-eye snapshot of the slot. Cold paths re-read this under the
/// cold lock (generation can't change while it is held).
#[inline]
pub(crate) fn status_for(&self, pid: Pid) -> Status {
self.word.status_for(pid.generation())
}
/// Does the slot currently hold the actor `pid` names, in a non-terminal
@@ -411,24 +394,7 @@ impl Slot {
/// CAS on the word.)
#[inline]
pub(crate) fn is_live_for(&self, pid: Pid) -> bool {
let w = self.word();
word_gen(w) == pid.generation()
&& !matches!(word_state(w), ST_VACANT | ST_DONE)
}
#[inline]
fn store_word(&self, gen: u32, st: u64) {
self.word.store(pack(gen, st), Ordering::Release);
}
#[inline]
fn cas_word(&self, gen: u32, from: u64, to: u64) -> Result<u64, u64> {
self.word.compare_exchange(
pack(gen, from),
pack(gen, to),
Ordering::AcqRel,
Ordering::Acquire,
)
self.status_for(pid) == Status::Live
}
fn store_closure(&self, c: Closure) {
@@ -449,43 +415,6 @@ impl Slot {
}
}
/// The unpark protocol — the one way anything outside the scheduler makes
/// an actor runnable. Returns `true` iff the caller must enqueue `pid`.
///
/// Loops on the packed word:
/// - generation mismatch → stale pid → no-op (atomic with the CAS: a
/// recycled slot can never be transitioned by an old pid).
/// - `Parked` → `Queued`: caller enqueues.
/// - `Running` → `RunningNotified`: the actor is inside a prep-to-park
/// window (registered somewhere, not yet parked). The scheduler's
/// park-return CAS will fail against this and re-queue immediately —
/// the lost-wakeup window is a state, not a flag.
/// - `Queued` / `RunningNotified`: someone already notified — coalesce.
/// - `Done` / `Vacant`: nothing to wake.
#[must_use]
pub(crate) fn unpark_action(&self, pid: Pid) -> bool {
let gen = pid.generation();
loop {
let w = self.word();
if word_gen(w) != gen {
return false;
}
match word_state(w) {
ST_PARKED => {
if self.cas_word(gen, ST_PARKED, ST_QUEUED).is_ok() {
return true;
}
}
ST_RUNNING => {
if self.cas_word(gen, ST_RUNNING, ST_RUNNING_NOTIFIED).is_ok() {
crate::te!(crate::trace::Event::UnparkDeferred(pid));
return false;
}
}
_ => return false, // Queued | RunningNotified | Done | Vacant
}
}
}
}
// ---------------------------------------------------------------------------
@@ -571,6 +500,18 @@ impl RuntimeInner {
/// into `Queued` (spawn's publish, the unpark protocol, or the
/// scheduler's yield/notified-park return paths).
pub(crate) fn enqueue(&self, pid: Pid) {
// Every push pairs 1:1 with a transition INTO Queued, and nothing can
// move the word off Queued until this very entry is popped — so the
// word must read EXACTLY (gen, Queued) here. This is the at-most-once-
// enqueued invariant the bounded rings' capacity proof leans on.
debug_assert_eq!(
self.slot_at(pid).map(|s| s.word.load()),
Some(crate::slot_state::pack(
pid.generation(),
crate::slot_state::ST_QUEUED
)),
"enqueue of a pid not in (gen, Queued)"
);
self.run_queue.push(pid);
crate::te!(crate::trace::Event::Enqueue(pid));
}
@@ -579,9 +520,15 @@ impl RuntimeInner {
/// The runtime-internal core of `scheduler::unpark`.
pub(crate) fn unpark(&self, pid: Pid) {
if let Some(slot) = self.slot_at(pid) {
if slot.unpark_action(pid) {
crate::te!(crate::trace::Event::UnparkDirect(pid));
self.enqueue(pid);
match slot.word.unpark(pid.generation()) {
Unpark::Enqueue => {
crate::te!(crate::trace::Event::UnparkDirect(pid));
self.enqueue(pid);
}
Unpark::Notified => {
crate::te!(crate::trace::Event::UnparkDeferred(pid));
}
Unpark::Noop => {}
}
}
}
@@ -797,7 +744,6 @@ pub(crate) fn install_actor(
let slot = &inner.slots[idx as usize];
let gen = slot.generation(); // stable: we own the vacant slot via the free list
let pid = Pid::new(idx, gen);
debug_assert_eq!(word_state(slot.word()), ST_VACANT);
let stop = Arc::new(AtomicBool::new(false));
slot.stop_ptr.store(Arc::as_ptr(&stop) as *mut _, Ordering::Release);
@@ -816,7 +762,7 @@ pub(crate) fn install_actor(
// Publish: only now can pops, unparks, or stops find the actor. The
// Release store orders everything above before any Acquire reader.
slot.store_word(gen, ST_QUEUED);
slot.word.publish_queued(gen);
inner.enqueue(pid);
crate::te!(crate::trace::Event::Spawn { parent: supervisor, child: pid });
pid
@@ -839,11 +785,7 @@ pub(crate) fn reclaim_slot(inner: &RuntimeInner, pid: Pid) {
let dropped_outside;
{
let mut cold = slot.cold.lock();
let w = slot.word();
if word_gen(w) != pid.generation()
|| word_state(w) != ST_DONE
|| cold.outstanding_handles != 0
{
if slot.status_for(pid) != Status::Done || cold.outstanding_handles != 0 {
return; // already reclaimed, or not yet eligible
}
debug_assert!(cold.actor.is_none(), "reclaiming a slot that still owns an actor");
@@ -859,7 +801,7 @@ pub(crate) fn reclaim_slot(inner: &RuntimeInner, pid: Pid) {
slot.stop_ptr.store(std::ptr::null_mut(), Ordering::Release);
// The generation bump IS the reclaim: every stale pid is dead from
// this store onwards (unpark protocol, pops, cold-path re-verifies).
slot.store_word(pid.generation().wrapping_add(1), ST_VACANT);
slot.word.reclaim(pid.generation());
}
drop(dropped_outside);
inner.free.lock().push(pid.index());
@@ -886,7 +828,6 @@ fn finalize_actor(inner: &Arc<RuntimeInner>, pid: Pid, outcome: Outcome) {
let slot = inner.slot_at(pid).expect("finalize_actor: pid out of range");
let (waiters, monitors, links, actor) = {
let mut cold = slot.cold.lock();
debug_assert_eq!(slot.generation(), pid.generation(), "finalize: slot vanished");
let actor = cold.actor.take().expect("finalize_actor: actor vanished");
cold.outcome = Some(joiner_outcome);
slot.stop_ptr.store(std::ptr::null_mut(), Ordering::Release);
@@ -894,7 +835,8 @@ fn finalize_actor(inner: &Arc<RuntimeInner>, pid: Pid, outcome: Outcome) {
// check-Done-or-register-waiter (also under it) can never miss: it
// either sees Done and takes the outcome, or its waiter registration
// happens before our take() below and is woken further down.
slot.store_word(pid.generation(), ST_DONE);
// (set_done self-asserts the Running|Notified precondition + gen.)
slot.word.set_done(pid.generation());
(
std::mem::take(&mut cold.waiters),
std::mem::take(&mut cold.monitors),
@@ -948,10 +890,7 @@ fn finalize_actor(inner: &Arc<RuntimeInner>, pid: Pid, outcome: Outcome) {
let trap = match inner.slot_at(peer) {
Some(ps) => {
let mut cold = ps.cold.lock();
let w = ps.word();
if word_gen(w) == peer.generation()
&& !matches!(word_state(w), ST_DONE | ST_VACANT)
{
if ps.status_for(peer) == Status::Live {
cold.links.retain(|p| *p != pid);
if abnormal {
Some(cold.actor.as_ref().and_then(|a| a.trap.clone()))
@@ -985,7 +924,8 @@ fn finalize_actor(inner: &Arc<RuntimeInner>, pid: Pid, outcome: Outcome) {
// monitor/trap sends, stop cascades) is enqueued before `live_actors`
// can be observed at its decremented value. See the termination note in
// the module docs.
inner.live_actors.fetch_sub(1, Ordering::Release);
let prev = inner.live_actors.fetch_sub(1, Ordering::Release);
debug_assert!(prev >= 1, "live_actors underflow — double finalize");
}
// ---------------------------------------------------------------------------
@@ -1165,11 +1105,7 @@ fn schedule_loop(inner: &Arc<RuntimeInner>, slot_idx: usize) {
Some(s) => s,
None => continue, // can't happen for real pids; defensive
};
if let Err(actual) = slot.cas_word(pid.generation(), ST_QUEUED, ST_RUNNING) {
debug_assert_ne!(
word_gen(actual), pid.generation(),
"queued pid in unexpected state {}", word_state(actual)
);
if !slot.word.try_claim(pid.generation()) {
continue; // stale pid: retry immediately (never the idle path)
}
crate::te!(crate::trace::Event::Dequeue(pid));
@@ -1214,26 +1150,18 @@ fn schedule_loop(inner: &Arc<RuntimeInner>, slot_idx: usize) {
// Running OR RunningNotified → Queued; a notification
// arriving mid-run coalesces into the re-queue.
crate::te!(crate::trace::Event::Yield(pid));
let prev = slot.word.swap(pack(gen, ST_QUEUED), Ordering::AcqRel);
debug_assert!(matches!(
word_state(prev), ST_RUNNING | ST_RUNNING_NOTIFIED
));
slot.word.yield_return(gen);
inner.enqueue(pid);
}
YieldIntent::Park => {
match slot.cas_word(gen, ST_RUNNING, ST_PARKED) {
Ok(_) => {
crate::te!(crate::trace::Event::Park(pid));
}
Err(actual) => {
// An unpark landed in the prep-to-park window:
// we are RunningNotified. Re-queue instead of
// parking — the lost-wakeup window, closed.
debug_assert_eq!(word_state(actual), ST_RUNNING_NOTIFIED);
slot.store_word(gen, ST_QUEUED);
crate::te!(crate::trace::Event::UnparkFlagConsumed(pid));
inner.enqueue(pid);
}
if slot.word.park_return(gen) {
crate::te!(crate::trace::Event::Park(pid));
} else {
// An unpark landed in the prep-to-park window; the
// word is back to Queued — re-queue instead of
// parking. The lost-wakeup window, closed.
crate::te!(crate::trace::Event::UnparkFlagConsumed(pid));
inner.enqueue(pid);
}
}
}