State machine extracted to src/slot_state.rs as a standalone unit (StateWord:
publish_queued / try_claim / yield_return / park_return / unpark / set_done /
reclaim, plus the Status view for cold paths). runtime.rs keeps the protocol
rationale and consumes the mechanism; every transition self-asserts its
precondition per the assert-the-invariants rule.
src/sync_shim.rs: std vs loom indirection (atomics + UnsafeCell with the
with/with_mut access API) for the two loom-modeled modules. Loom models run
the PRODUCTION code, not a replica:
- slot_state: no-lost-wakeup (park vs unpark), two-unparkers-one-enqueue,
stale-unpark-never-hits-reused-slot (the ABA theorem), unpark-vs-claim.
- run_queue: mpmc exactly-once through a lap wraparound, push/pop race,
striped two-producer drain.
RUSTFLAGS="--cfg loom" cargo test --lib --release — 7 models, all pass.
RawMutex deliberately not loom-modeled (futexes can't be; textbook mutex3
with stress + unwind coverage).
Assertion sweep (invariants now checked at the point of reliance):
- enqueue debug-asserts the word reads EXACTLY (gen, Queued) — the
at-most-once-enqueued invariant the ring capacity proof leans on.
- RawMutex enforces the leaf rule mechanically: debug-build thread-local
held-count, panics at the acquisition that violates it.
- live_actors underflow (double finalize) asserted.
- StateWord transition preconditions asserted (yield/park/done/reclaim/
publish/claim).
Audit: with_runtime, RawMutex guards, run-queue ops, and trace::record all
gate preemption (and thereby the stop sentinel) for their span; trace was
already self-gating. Sole remaining exception is the channel std MutexGuard
— the documented first post-v0.5 fast-follow.
Review fixes to the branched-in work:
- slot_state::status_for mapped (matching gen, Vacant) to Stale instead of
unreachable!: Pid::new is public, so a forged/never-issued pid (e.g.
monitor(Pid::new(5,0)) on a fresh slab) could reach it — "no such actor"
is the correct total answer; issued pids still can't get there.
- Tightened enqueue's assert from Status::Live to the exact (gen, Queued)
word its own comment argues for.
Validated: 22 suites in debug (all asserts + leaf counter live) for
rq-mutex/rq-mpmc/rq-striped, release for rq-mutex, smarm-trace build +
stress, loom 7/7.
- benches/rq_micro.rs: raw-structure microbench, threads x producer:consumer
ratio sweep. Benches all three queue types in one binary (they compile in
every build; only the runtime alias is feature-selected), so no rebuild
dance. Queues sized to the op count so the occupancy contract is met.
- benches/rq_runtime.rs: whole-runtime benches with the selected variant:
yield-storm (pure queue churn), ping-pong-pairs (park/unpark latency),
spawn-storm (slab + free list + queue under churn). Scheduler-count sweep.
- scripts/bench_rq.sh: rebuilds rq_runtime per rq-* feature, runs rq_micro
once, aggregates RQCSV lines into bench_results/summary.csv.
- All knobs via SMARM_BENCH_* env vars; house table format + machine lines.
- run_queue module is now #[doc(hidden)] pub (types + push/pop/len +
MpmcRing::with_capacity) solely so the external bench binary can drive the
raw structures.
docs(roadmap): phase 4 ticked (harness done; numbers from the 20-core box).
New fast-follow per review: assert the invariants we lean on — debug_assert!
on hot paths, loud assert!/panic on cold ones, at the point of reliance;
sweep existing code during the phase-5 audit, adopt as house style.
Validated end-to-end at smoke scale on the 1-core sandbox: full driver run,
24-row summary.csv across micro (3 structures x sweeps) and runtime
(3 variants x 3 benches x thread sweep).
src/run_queue.rs: the run queue extracted behind a compile-time-selected
type alias; mutually-exclusive cargo features with compile_error! guards
(zero or >1 selected). No runtime dispatch. All variants compile in every
build so their unit tests always run; the feature only picks the alias.
- rq-mutex (default): Mutex<VecDeque>, the control/baseline.
- rq-mpmc: hand-rolled Vyukov bounded MPMC ring, per-cell sequence numbers,
padded enqueue/dequeue counters. Strict FIFO, lock-free.
- rq-striped: M Vyukov rings (M = thread_count rounded up to pow2),
fetch-add ticket distribution, probe-from-home on push. Relaxed FIFO with
stripe-bounded skew; Σcapacity ≈ 2×max_actors so the probe terminates.
Capacity soundness: occupancy ≤ max_actors by the at-most-once-enqueued
invariant + slab cap, rings sized ≥ that bound, so push is infallible; a
full ring panics as a double-enqueue invariant violation rather than spin.
Two contracts the extraction made explicit (documented + debug-asserted):
- Queue ops require preemption disabled: a producer suspended between
claiming a cell and publishing its sequence stalls every consumer behind
it — livelock, since the suspended actor's own resume entry is behind the
hole. Structurally guaranteed since the phase-2 with_runtime NoPreempt fix.
- pop()==None is a snapshot, not a fence. Termination is counter-first:
every queue entry's target stays Queued (hence live) until that entry is
popped, so live==0 alone implies nothing actionable is or can be queued;
argument rewritten at the schedule_loop site. SharedState and with_shared
are deleted — nothing global is mutex-guarded on the run path under the
ring variants.
Validated: 22 suites green per variant (release for all three; debug with
live asserts for all three), ring unit tests (FIFO, lap wraparound,
4p/4c exactly-once, skewed drain) in every build, both compile_error!
guards verified to fire.
While running benchmarks, a hang surfaced in timer-only workloads — actors
sleeping with no IO in flight. Tracing it down, the race lives in sleep():
between the call to `timers.insert_sleep` and the subsequent `park_current`
yield, the actor is still in State::Runnable. If the timer fires in that
window, the old code's `if matches!(slot.state, State::Parked)` guard silently
drops the wakeup. The actor then parks normally and never gets re-queued —
it sleeps forever.
The fix mirrors what scheduler::unpark() and the IO FdReady path already do:
when the timer fires and the slot is still Runnable, set `pending_unpark`
instead of re-queuing immediately. The upcoming Park yield sees the flag and
re-queues the actor rather than suspending it, closing the race without any
new synchronisation.
Adds a regression test: 100 actors doing pure timer sleeps across ≥2
scheduler threads. The test asserts both correctness (all actors complete)
and timeliness (wall time < 2×sleep duration), which is enough signal to
catch a stuck actor even on a single-core CI runner.
Three changes, each independently measured, landed together.
Fuse pre-resume mutex acquisitions
-----------------------------------
The schedule loop previously took the shared lock three separate times
per actor resume: once to pop the run queue, once to read the stack
pointer, and once to pop the first-resume closure. These are now a
single acquisition that returns everything needed to resume the actor.
As a side effect, pending_closures was changed from a Vec<(Pid, Closure)>
with O(n) linear scan to a Vec<Option<Closure>> indexed by slot index,
making first-closure lookup O(1).
Move stack allocation outside the shared lock
----------------------------------------------
Stack::new() (mmap + mprotect) was previously called inside with_shared,
stalling every other scheduler thread for the duration of two syscalls on
every spawn. It now runs before the lock is acquired.
Stack pool
----------
Rather than munmap-ing a stack when an actor finishes and mmap-ing a fresh
one on the next spawn, stacks are now recycled through a per-Runtime pool
(Mutex<Vec<Stack>>). finalize_actor extracts the stack from the Actor
before clearing the slot and pushes it to the pool outside the shared lock.
spawn_under pops from the pool before falling back to Stack::new().
The pool is unbounded for now (shrink policy TBD) but capped at
stack_pool_cap stacks on return, defaulting to thread_count * 4.
The cap is configurable via Config::stack_pool_cap(n).
Results (24-thread, against stored baseline)
--------------------------------------------
chained_spawn smarm 1-thread: 9763 → 261 µs (-97%)
chained_spawn smarm 24-thread: 23562 → 838 µs (-96%)
ping_pong_oneshot smarm 1-thread: 18409 → 742 µs (-96%)
ping_pong_oneshot smarm 24-thread: 44596 → 1425 µs (-97%)
catch_unwind_panics smarm 24-thread: 267812 → 124094 µs (-54%)
fan_out_compute smarm 24-thread: 2839 → 2226 µs (-22%)
Tokio regressions in the checker output are baseline measurement drift;
no tokio code was changed.
Add `Config::alloc_interval()` and `Config::timeslice_cycles()` so
callers can tune preemption sensitivity at runtime. The values flow
through `RuntimeInner` and are written into per-scheduler-thread locals
via a new `configure_preempt()` call at thread startup, keeping the hot
path free of cross-thread coherency traffic.
Fix unused-variable warnings in channel.rs by inlining `current_pid()`
directly into `te!` macro arguments — since the no-op macro arm never
evaluates its argument, no binding is needed at the call site.
Clean up a handful of dead imports exposed by the refactor.
Hand-rolled context switching on mmap'd stacks with guard pages,
allocator-driven RDTSC preemption, unbounded MPSC channels, supervision
via per-slot Signal mailboxes, root supervisor as sentinel PID.
Lib + tests + benches clean check/clippy. All 29 tests pass.
Bench: smarm 3.4% over serial baseline, within 160us of tokio
current-thread on prime-counting fan-out.
Claude