Replace implicit unwrap()/expect() in the lock-ordered core with explicit
match arms. Lock-poison sites use one uniform message
("smarm: <lock> lock poisoned (core corrupt): {e}"); invariant sites panic
with a descriptive message naming the violated invariant. No behaviour
change: each rewrite preserves the prior panic-on-bad-arm semantics. Also
clears the accompanying clippy hygiene in these files (redundant_closure,
len_without_is_empty, too_many_arguments, unnecessary_sort_by,
missing_safety_doc, nonminimal_bool/unnecessary_unwrap).
A thread-local Cell<Option<Pid>> per scheduler, checked before the shared
run queue. Runtime-selected via Config { wake_slot: bool }, default OFF
until the slot shootout accepts it (one binary benches both arms).
Push policy: slot-eligible iff the wake originates from actor context
(current_pid().is_some()) — the slot push replaces run_queue.push at the
tail of the unpark protocol's Parked → Queued CAS, so at-most-once-enqueued
holds verbatim as (slot ⊕ shared queue). Scheduler-context wakes (timer/IO
drain) and spawns always go shared (spawns never reach unpark_inner at all).
Displacement is Go semantics: newest wake takes the slot, occupant pushed
shared — moved, never copied.
Pop order: slot, then shared. Slot-popped actors skip reset_timeslice() and
inherit the waker's remaining slice; a handoff chain is bounded by one
slice, after which the preempt-yield re-enqueue goes shared (a yield is not
a wake) — the one-slice starvation bound, zero new counters. Idle and
AllDone are only reachable with an empty local slot by pop order; an
occupied slot elsewhere holds a Queued (live) actor, so the counter-first
termination argument is untouched.
Observability: per-thread slot_hits / slot_displacements (reset at run()
start so post-run stats() reads are per-run), SlotPush/SlotPop trace events.
Bench plan (roadmap v0.9 item 2): rq_runtime gains the slot on/off
dimension (SMARM_BENCH_SLOT, default "0 1") — ping-pong-pairs is the
target metric, yield-storm the regression guard, spawn-storm the
neutrality check. RQCSV grows a slot column; RQSLOT lines carry the
counters; bench_rq.sh aggregates both. Tests pin the push policy through
the counters (actor-context hits, spawn/join bypass, displacement,
default-off, per-run reset).
State machine extracted to src/slot_state.rs as a standalone unit (StateWord:
publish_queued / try_claim / yield_return / park_return / unpark / set_done /
reclaim, plus the Status view for cold paths). runtime.rs keeps the protocol
rationale and consumes the mechanism; every transition self-asserts its
precondition per the assert-the-invariants rule.
src/sync_shim.rs: std vs loom indirection (atomics + UnsafeCell with the
with/with_mut access API) for the two loom-modeled modules. Loom models run
the PRODUCTION code, not a replica:
- slot_state: no-lost-wakeup (park vs unpark), two-unparkers-one-enqueue,
stale-unpark-never-hits-reused-slot (the ABA theorem), unpark-vs-claim.
- run_queue: mpmc exactly-once through a lap wraparound, push/pop race,
striped two-producer drain.
RUSTFLAGS="--cfg loom" cargo test --lib --release — 7 models, all pass.
RawMutex deliberately not loom-modeled (futexes can't be; textbook mutex3
with stress + unwind coverage).
Assertion sweep (invariants now checked at the point of reliance):
- enqueue debug-asserts the word reads EXACTLY (gen, Queued) — the
at-most-once-enqueued invariant the ring capacity proof leans on.
- RawMutex enforces the leaf rule mechanically: debug-build thread-local
held-count, panics at the acquisition that violates it.
- live_actors underflow (double finalize) asserted.
- StateWord transition preconditions asserted (yield/park/done/reclaim/
publish/claim).
Audit: with_runtime, RawMutex guards, run-queue ops, and trace::record all
gate preemption (and thereby the stop sentinel) for their span; trace was
already self-gating. Sole remaining exception is the channel std MutexGuard
— the documented first post-v0.5 fast-follow.
Review fixes to the branched-in work:
- slot_state::status_for mapped (matching gen, Vacant) to Stale instead of
unreachable!: Pid::new is public, so a forged/never-issued pid (e.g.
monitor(Pid::new(5,0)) on a fresh slab) could reach it — "no such actor"
is the correct total answer; issued pids still can't get there.
- Tightened enqueue's assert from Status::Live to the exact (gen, Queued)
word its own comment argues for.
Validated: 22 suites in debug (all asserts + leaf counter live) for
rq-mutex/rq-mpmc/rq-striped, release for rq-mutex, smarm-trace build +
stress, loom 7/7.
- benches/rq_micro.rs: raw-structure microbench, threads x producer:consumer
ratio sweep. Benches all three queue types in one binary (they compile in
every build; only the runtime alias is feature-selected), so no rebuild
dance. Queues sized to the op count so the occupancy contract is met.
- benches/rq_runtime.rs: whole-runtime benches with the selected variant:
yield-storm (pure queue churn), ping-pong-pairs (park/unpark latency),
spawn-storm (slab + free list + queue under churn). Scheduler-count sweep.
- scripts/bench_rq.sh: rebuilds rq_runtime per rq-* feature, runs rq_micro
once, aggregates RQCSV lines into bench_results/summary.csv.
- All knobs via SMARM_BENCH_* env vars; house table format + machine lines.
- run_queue module is now #[doc(hidden)] pub (types + push/pop/len +
MpmcRing::with_capacity) solely so the external bench binary can drive the
raw structures.
docs(roadmap): phase 4 ticked (harness done; numbers from the 20-core box).
New fast-follow per review: assert the invariants we lean on — debug_assert!
on hot paths, loud assert!/panic on cold ones, at the point of reliance;
sweep existing code during the phase-5 audit, adopt as house style.
Validated end-to-end at smoke scale on the 1-core sandbox: full driver run,
24-row summary.csv across micro (3 structures x sweeps) and runtime
(3 variants x 3 benches x thread sweep).
src/run_queue.rs: the run queue extracted behind a compile-time-selected
type alias; mutually-exclusive cargo features with compile_error! guards
(zero or >1 selected). No runtime dispatch. All variants compile in every
build so their unit tests always run; the feature only picks the alias.
- rq-mutex (default): Mutex<VecDeque>, the control/baseline.
- rq-mpmc: hand-rolled Vyukov bounded MPMC ring, per-cell sequence numbers,
padded enqueue/dequeue counters. Strict FIFO, lock-free.
- rq-striped: M Vyukov rings (M = thread_count rounded up to pow2),
fetch-add ticket distribution, probe-from-home on push. Relaxed FIFO with
stripe-bounded skew; Σcapacity ≈ 2×max_actors so the probe terminates.
Capacity soundness: occupancy ≤ max_actors by the at-most-once-enqueued
invariant + slab cap, rings sized ≥ that bound, so push is infallible; a
full ring panics as a double-enqueue invariant violation rather than spin.
Two contracts the extraction made explicit (documented + debug-asserted):
- Queue ops require preemption disabled: a producer suspended between
claiming a cell and publishing its sequence stalls every consumer behind
it — livelock, since the suspended actor's own resume entry is behind the
hole. Structurally guaranteed since the phase-2 with_runtime NoPreempt fix.
- pop()==None is a snapshot, not a fence. Termination is counter-first:
every queue entry's target stays Queued (hence live) until that entry is
popped, so live==0 alone implies nothing actionable is or can be queued;
argument rewritten at the schedule_loop site. SharedState and with_shared
are deleted — nothing global is mutex-guarded on the run path under the
ring variants.
Validated: 22 suites green per variant (release for all three; debug with
live asserts for all three), ring unit tests (FIFO, lap wraparound,
4p/4c exactly-once, skewed drain) in every build, both compile_error!
guards verified to fire.
smarm-agent