- Compact v0.9 (Wake-path latency) into Shipped; record the RFC 004 spinning
excision as a deviation (preserved on branch rfc-004-spinning).
- Drop the v0.7 entry, folding its reference into the Shipped history pointer.
- Promote Process groups from Later into the up-next slot v0.9 vacated;
Per-switch cost now leads Later/Highest priority.
The spinning experiment is excised from master. Drops the two dedicated v0.9
sections describing it. Inline RFC-004 mentions in the v0.9 intro and the
queue-topology decision record are intentionally left for a manual short-term
-plan pass.
Carries forward the non-spin parts of the dropped spin-tooling commit:
- .gitignore: __pycache__/ and *.pyc
- sweep.py: generic vs-tokio comparison summary (no spin_sweep coupling)
The spin_sweep README docs from that commit are intentionally left behind.
The profile was captured against the spin-enabled build. With the RFC 004
spinning experiment excised from master, the ~12% hot-path futex_wake it
attributed to the spinning submit-rule no longer exists (idle wait is back to
thread::sleep). Add a provenance note, mark the futex_wake row/finding as
excised spin machinery, and drop the now-moot 'gate the submit wake' lead.
Spin-independent findings (shims ~1%, schedule_loop + run-queue ~33%) unchanged.
The terminal-wake fix cannot reach the (deadline, no-io) idle branch:
thread::sleep, no pipe to write. Documented as a Later item with the
two candidate mechanisms (clamped sleep vs condvar/futex), plus the
cross-thread-unpark observation from the same session.
request_stop sets the flag and issues a wildcard unpark, but unpark
no-ops on a Queued actor (the pending run "is" the wake). If the
actor's first action on resume is a blocking park — no allocation, no
check!() on the way — the wake-side-only check in park_current is
unreachable: the actor parks with the stop flag already set and no
wake is ever coming. The runtime then idles forever.
Fix: check_cancelled at the ENTRY of park_current, before the switch.
The remaining window (flag lands after the entry check, before the
park) is covered by the existing protocol: the stop's unpark finds
Running / the prep-to-park window, sets Notified, and the park-return
re-queues into the wake-side check. Unwinding from the entry is the
same unwind path as the wake side; leftover wait registrations are
stale-epoch / dead-generation and self-clean at their wakers' failed
CAS.
This is bug #1 of the urus chunk-2 session (bug #2, the terminal
wake, is the preceding commit). Repro: tests/cancel.rs
stop_flagged_while_queued_lands_at_first_park — deadlocks without the
fix (watchdog-bounded), passes in <50ms with it.
An idle scheduler thread blocks in poll_wake on a snapshot of
peek_deadline / io_outstanding. Only the io threads ever write the wake
pipe (on completion push); enqueue does not. Mid-flight that is masked —
the thread that caused an enqueue is awake and processes it — but at
termination the snapshot can go terminally stale, two ways:
- Stale io_outstanding (infinite hang): an actor parked in wait_readable
is request_stop'ped. Cancellation deregisters the waiter and produces
NO completion, so the wake pipe is never written. A sibling blocked on
io_outstanding > 0 with no timers pending sits in poll(-1) forever.
- Orphaned timer deadline (finite stall): an actor cancelled out of a
long sleep leaves its timer entry behind. Clearing the wheel at
AllDone doesn't help a sibling that already blocked on that deadline;
it sleeps it out in full.
In both cases the remaining actors finish on the other scheduler thread,
which reaches the live==0 && io_out==0 verdict and returns without
waking the blocked one. Runtime::run then stalls or hangs in its worker
join. Found via urus's graceful-shutdown tests (~10% flake, one test per
variant); confirmed by gdb dumps of hung processes showing run() in
JoinHandle::join over a sibling in poll(wake_fd, 59750ms) resp. poll(-1).
Fix: Io::wake() writes the wake pipe directly; the AllDone arm calls it
after the timer clear. One byte wakes every poller; each re-runs the
verdict, independently reaches AllDone, and re-wakes — idempotent.
tests/terminal_wake.rs reproduces both variants deterministically: the
root busy-spins (no timer entries, occupies one scheduler thread) so the
sibling settles into the stale idle wait before the stop is issued. Both
hang without the fix and pass in <0.5s with it.
Known residual gap, deliberately unfixed: the timers-pending /
no-io-subsystem idle branch blocks in thread::sleep and has no wake
mechanism at all — the same stall exists for runtimes that never
initialize io. Roadmap candidate alongside cross-thread unpark and
entry-side check_cancelled in park_current (the lossy-QUEUED-stop bug).
Replace single-process run_benches() with a two-layer approach:
- run_benches_once(): one cargo bench invocation with SMARM_BENCH_SETS=1,
yielding one median per label from ITERS raw samples.
- run_benches(): calls run_benches_once() BENCH_SETS=5 times, accumulates
the per-set medians as independent samples, and returns the median of
those 5 values.
Each set is a fully isolated process (fresh PID, cold caches, independent
OS scheduler context, new warmup run), so the 5 samples are statistically
independent. Correlated noise within a single sustained process run — CPU
frequency scaling, sticky OS scheduler decisions, warm branch predictors —
no longer contaminates the final median.
The previous fix (SMARM_BENCH_SETS loop inside run_n) collected more
samples but within one process, giving correlated samples that were still
vulnerable to a noisy measurement window for any one label.
Each run_n call now collects ITERS × SETS raw samples across all sets
(one warmup before the first set, discarded), then takes a single global
median. Previously only ITERS=15 samples were taken per bench invocation,
which produced noisy results in sweep.py regress.
SMARM_BENCH_SETS is read from the environment; the default of 5 gives
75 samples per bench row (5×15), matching the stability of the multi-run
bash approach without requiring multiple cargo bench invocations.
Also moves scripts/bench_rq.sh into benches/ alongside sweep.py.
The cd "$(dirname "$0")/.." path logic is unchanged.
Adding slot_hits + slot_displacements in RFC 005 grew SchedulerStats
from 16 to 32 bytes — exactly 2 per cache line. run_queue_len (written
on every push/pop by thread N) then false-shared with fields of thread
N+1, causing ~45-60% yield-storm regression at t≥8 visible equally in
slot-on and slot-off paths.
A thread-local Cell<Option<Pid>> per scheduler, checked before the shared
run queue. Runtime-selected via Config { wake_slot: bool }, default OFF
until the slot shootout accepts it (one binary benches both arms).
Push policy: slot-eligible iff the wake originates from actor context
(current_pid().is_some()) — the slot push replaces run_queue.push at the
tail of the unpark protocol's Parked → Queued CAS, so at-most-once-enqueued
holds verbatim as (slot ⊕ shared queue). Scheduler-context wakes (timer/IO
drain) and spawns always go shared (spawns never reach unpark_inner at all).
Displacement is Go semantics: newest wake takes the slot, occupant pushed
shared — moved, never copied.
Pop order: slot, then shared. Slot-popped actors skip reset_timeslice() and
inherit the waker's remaining slice; a handoff chain is bounded by one
slice, after which the preempt-yield re-enqueue goes shared (a yield is not
a wake) — the one-slice starvation bound, zero new counters. Idle and
AllDone are only reachable with an empty local slot by pop order; an
occupied slot elsewhere holds a Queued (live) actor, so the counter-first
termination argument is untouched.
Observability: per-thread slot_hits / slot_displacements (reset at run()
start so post-run stats() reads are per-run), SlotPush/SlotPop trace events.
Bench plan (roadmap v0.9 item 2): rq_runtime gains the slot on/off
dimension (SMARM_BENCH_SLOT, default "0 1") — ping-pong-pairs is the
target metric, yield-storm the regression guard, spawn-storm the
neutrality check. RQCSV grows a slot column; RQSLOT lines carry the
counters; bench_rq.sh aggregates both. Tests pin the push policy through
the counters (actor-context hits, spawn/join bypass, displacement,
default-off, per-run reset).
FdArm composes fd readiness with channel arms on one wait epoch.
Selectable grows fallible sel_register and an eager-cleanup hook;
losing/stop-unwound/timed-out fd arms are unregistered (waiters entry
+ kernel ONESHOT) so the fd is never poisoned. try_select /
try_select_timeout surface registration errors (EBADF, EMFILE,
AlreadyExists) instead of RFC 008's permanently-ready lean, which
would busy-loop a healthy-but-unregistrable fd; select/select_timeout
stay infallible for channel-only arms. Adds wait_readable_timeout /
wait_writable_timeout as one-arm selects.
Known benign race (pre-existing, slightly widened): a queued FdReady
racing the cleanup DEL can spuriously wake a fresh waiter on that fd;
absorbed by select's defensive re-loop. Fixable by epoch-stamping
completions.
Shootout (6d9f369 harness, 24-core sweep) landed in RFC 005's World 3:
queue variants indistinguishable in rq_runtime; per-wake latency is the
ceiling. Record the queue-topology closure (rq-mutex frozen as default,
benchmark-driven reopening only), define v0.9 as the wake-path latency
cycle (RFC 005 slot -> bench -> RFC 004 spinning workers -> bench +
interaction pass), and add per-switch cost to Later pending a profiling
spike + RFC.
Completed cycles compacted to the two trailing minors (v0.7, v0.8);
earlier history lives in git.
- Replace v0.3 content throughout; bump version label to v0.8
- New SVG: Krebs-cycle scheduler loop (6 stations, actor handoff as
energy exchange, finalize leaving the cycle)
- New SVG: gen_server dual-plane (model vs implementation mapping)
- New SVG: slot state machine with packed word, park-epoch and all
transitions (loom theorems annotated)
- Updated SVG: module map (22 modules, 4 layers incl. slot_state,
run_queue, raw_mutex, sync_shim, monitor, link, registry, trace)
- Updated SVG: dep graph (runtime hub, OTP layer on public surface)
- New sections: Slot State & Park-Epoch, Run Queue variants,
GenServer, OTP (monitors/links/supervisors/registry)
- Updated: init (slab allocation, counter-based termination), spawn
(closure in slot AtomicPtr, stack pool, live_actors ordering),
preemption (stop sentinel + StopSentinel/Outcome::Stopped),
IO (epoch-matched completions, fd-leak fix), gotchas (lost-wakeup
and global-mutex flipped to green; 4 new cards)
- Yield-sources table grown to 9 rows (select, recv_timeout,
call, request_stop)
- Fix: swap local css2.css font ref for Google Fonts CDN
A stopped actor unwinding out of wait_fd's park leaked its waiters entry
and the kernel-side EPOLLONESHOT registration; the stale entry then failed
every future wait_*() on that fd with AlreadyExists (the defensive bare DEL
in epoll_register sits behind the contains_key check, so it never ran).
Fix where the invariant breaks: a drop guard in wait_fd, armed after a
successful register and forgotten on the normal wake (where FdReady already
removed + DEL'd). On unwind it cleans up iff the entry is still this wait's
(pid, epoch) — an entry consumed by a racing FdReady means the fd may carry
another actor's fresh registration, which must be left alone. Closes the
v0.2 fd-hygiene TODO.
Monitors are created at runtime (you watch a worker you just spawned in a
handler), so down arms can't ride the static info list. init grows a
&ServerCtx parameter (breaking; no-op default) whose clonable Watcher hands
Monitors to the loop over a control arm; the loop selects the live down
arms ahead of everything else. Arm priority: downs → control → infos →
inbox. A delivered Down retires its arm (monitors are one-shot); a state
that never clones the Watcher closes the control arm after init and the
loop falls back to the plain-inbox park.
type Info + handle_info (no-op default) on GenServer; ServerBuilder
(with_info/under/start) so start variants don't multiply — start/start_under
stay as wrappers. The loop selects [infos.., inbox] in priority order when
info arms exist, and keeps the plain recv() park when none do. Closed info
arms are silently removed (closed-arm-is-ready-forever); a closed inbox
still means graceful shutdown.
Gotchas now lead with the consuming-wake invariant ('the only wildcard wake
is request_stop') and the obligations it puts on future wakers. Deviations
recorded: the no-park exit's retire_wait/clear_notify (a protocol piece the
plan missed), Drop-guard deregistration superseded by own-pid assert
relaxation, and closed arms being ready forever.
The deadline is one more stamped waker on the select's wait epoch: a unit
TimerTarget whose on_timeout is a bare unpark_at. Nothing registers in any
arm for it, so there is nothing to cancel or leak — an arm winning leaves
the timer entry to die at its epoch CAS; the timer winning leaves the arms'
registrations to self-clean like any select loser's. Classification is pure
channel state (wakes are precise): some arm ready -> Some(first, priority
order); none -> the timer was the only remaining stamped waker -> None.
Message-first on a raced deadline, same as recv_timeout.
Registration pass factored into register_arms, which owns the retire_wait
obligation on the ready-now exit for both entry points.
select(&[&dyn Selectable]) -> usize registers (pid, epoch) in every arm
under one wait epoch, parks once, and returns the first ready index in
documented priority order (BEAM-style; no fairness promise). A closed arm
counts as ready — and stays ready forever, so callers drop it from the set
once observed. Losing arms need no cancellation pass: the winning wake
consumed the epoch, so their registrations die at their wakers' failed CAS
or are overwritten by the receiver's next wait on that channel (the
single-receiver debug_asserts relax to 'none or own pid' accordingly).
The one genuinely new protocol piece is the no-park exit: returning with an
arm ready at registration time leaves earlier arms holding LIVE-epoch
registrations, whose wakes could fault a later one-shot park as a pending
notification. scheduler::retire_wait closes it — bump the epoch (in-flight
wakes die at their CAS), eat a notification that already landed
(StateWord::clear_notify), then re-observe the stop flag, in that order.
Proved by the new loom theorem retire_eats_late_arm_notification; the
integration probes in tests/select.rs fire stale loser-arm wakes and assert
a subsequent sleep's one-shot park holds its full duration.
The slot epoch is THE wait identity, so the hand-rolled per-primitive copies
go away: channel loses cur_wait/next_wait_seq/timed_out, mutex loses Wait.seq
and next_seq, TimerTarget::on_timeout takes the epoch. Registrations become
(pid, epoch) — channel parked_receiver, mutex waiters, io fd waiters,
Blocking io completions, sleep timers, joiner lists — and their wakers move
to unpark_at. begin_wait is lock-free, so each primitive opens the wait
inside the same critical section that publishes the registration.
recv_timeout's wake-classification loop collapses: wakes are precise, so
queue → Ok, senders==0 → Disconnected, else Timeout — the 'Defensive'
re-register branch is now unreachable by protocol, not by audit. Same for
Mutex::lock_timeout's one-shot park.
End-state invariant, auditable in one sentence: the only wildcard wake is
request_stop, which is terminal.
Word repacks to (gen << 32) | (epoch << 8) | state. begin_wait opens a wait
identity; every successful wake consumes it, so at most one wake lands per
wait by construction. unpark grows an Option<epoch> match; RuntimeInner gains
unpark_at. Claim/yield/park transitions become epoch-preserving CAS loops.
Loom: all four prior theorems re-proved on the new word, plus
consumed_epoch_unpark_never_lands — a late waker stamped with a consumed
epoch can neither enqueue nor notify, in every interleaving (the property
select's loser arms and every one-shot park site lean on).
No callers stamped yet; behaviour identical. Two transient dead_code
warnings (begin_wait, unpark_at) are consumed by the next commit.
Marks the three v0.6 items done with their hashes and the load-bearing
deviations: two-class lock order replacing the strict leaf rule, the
registry as a bimap with name_of and without send_by_name (no per-pid
mailbox to send to), and call_timeout built on recv_timeout instead of
monitor machinery. Adds the Leaf -> Channel order to the standing
invariants; README module table picks up recv_timeout, call_timeout,
and the registry row.
Deviates deliberately from the roadmap's monitor-based sketch
(monitor + wait reply-or-Down-or-deadline + demonitor): server death is
already observable on the reply channel itself - the reply sender drops
as the server unwinds, closing the channel and waking the parked caller.
So a bounded call is exactly recv_timeout on the reply channel, mapping
Disconnected -> ServerDown and Timeout -> Timeout. No registration
exists, so a timed-out call leaks nothing by construction - the
guarantee the monitor design had to engineer.
Semantics on timeout match Erlang: the request stays in the inbox and
is still handled; the late reply's send fails harmlessly against the
dropped reply receiver. CallTimeoutError keeps ServerDown and Timeout
distinguishable; the infallible call() is unchanged.
Tests: reply within deadline, timeout against a slow (parking) handler
with elapsed bounds, server survival across an abandoned call (late
reply discarded, bounded and unbounded calls keep working), and
ServerDown-not-Timeout for both death paths (mid-call panic and
already-gone inbox).
The per-recv deadline the roadmap deferred, built exactly the way
Mutex::lock_timeout already works: register the wait with a per-wait
seq, arm a timer::Reason::WaitTimeout with the channel inner (now a
TimerTarget) as the target, park. On expiry on_timeout cancels the wait
only if that same seq is still parked; satisfied or abandoned waits
leave a stale heap entry that no-ops on seq mismatch, per the
no-cancellation convention in timer.rs.
Race resolution is message-first: a send that lands by the time the
woken receiver runs is delivered even if the deadline had also passed.
Closure is reported as RecvTimeoutError::Disconnected, keeping timeout
and server/sender death distinguishable for callers (gen_server call
timeout builds on exactly this distinction next).
The timer is armed outside the channel critical section (the timers
lock must never nest under a Channel-class lock); the unpark race this
opens is absorbed by the RunningNotified protocol.
Tests: immediate delivery, actual timeout (with elapsed check), prompt
wake on send, Disconnected on close, zero-duration poll, post-timeout
seq isolation (stale entry must not cancel later waits), and a
4-scheduler mixed-outcome run (12 fed / 12 timed out).
register/whereis/unregister plus the inverse lookup name_of. Erlang
semantics: one name per pid, one pid per name, registering over a live
binding errors; same-binding re-register is an idempotent Ok.
Bimap = two HashMaps held in exact inverse under one Leaf RawMutex in
RuntimeInner; the invariant is debug-asserted on every mutation. The
inverse direction costs one extra String per binding and buys O(1)
pid->name for supervisors/tracing/diagnostics.
Cleanup is lazy: no finalize_actor hook, no Slot field (which would buy
into the reset-in-three-places invariant). Liveness rides the
generation-checked slot word - pids are never reused, so a stale binding
is detectable, never misdirected - and bindings to dead actors behave as
absent, pruned on contact by whereis/name_of/register/unregister.
Liveness checks under the registry lock read only the atomic slot word,
so the leaf rule holds trivially.
send_by_name is deliberately absent: smarm has no per-pid mailbox, so
there is nothing generic to send *to* - the registry yields a Pid,
usable with monitor/link/request_stop and as routing for user channels.
Tests: roundtrip both directions, idempotence, NameTaken on live holder,
one-name-per-pid, NoProc, death-evaporates-binding (via lookup pruning
and via register's own eviction path), unregister, cross-actor lookup
wired into request_stop, and a 4-scheduler name-contention churn test.
Phase 2 surfaced the hole this closes: channel guards were held with
preemption enabled, so a timeslice switch inside a channel critical
section could release the pthread mutex from a different OS thread (UB).
RawMutex disables preemption for the guard span and is
cross-thread-release sound by construction; poison goes away with it.
The strict single-class leaf rule cannot survive the migration: finalize
clones the supervisor/trap senders and monitor() clones the Down sender,
all under a cold lock, and those senders live in the slot - the nesting
is structural. The leaf check is therefore generalized to two classes:
Leaf (cold locks, free list, stack pool) and Channel. Order is
Leaf -> Channel, at most one of each; both directions of violation are
debug-asserted at the acquisition site. Channel critical sections call
only the lock-free unpark protocol, so the order is acyclic.
Tests: class-ordering unit tests in raw_mutex (allowed nesting + all
three rejected shapes), plus a multi-scheduler integration test driving
channels through monitor churn and actor death so any ordering
regression trips the debug assert instead of deadlocking.
benches/README.md: catalog of the two bench families (cross-runtime
comparisons + the v0.5 run-queue shootout), how to run the shootout
(scripts/bench_rq.sh, SMARM_BENCH_* knobs, the --no-default-features dance,
summary.csv/RQCSV format), honest-numbers caveats (core count is the
experiment; striped losing at low thread counts is expected), and the
conventions for adding a bench.
tests/README.md: catalog grouped by layer (low-level units / feature areas /
regression+stress), the run matrix (debug-first — that's where the invariant
asserts live — three queue variants, release, loom, trace), why loom tests
live in src/ rather than tests/, and the house conventions: one runtime per
test, oversubscription on purpose, regression tests validated against the
reintroduced bug, odds-stacking for stochastic tests, ordering-not-duration
time assertions, and new-invariant = assert + loom model.
Also fixes a stale header in tests/mutex.rs that claimed 'loom::Mutex' —
it tests smarm::Mutex, and the old name is actively confusing now that loom
is real in this repo.
State machine extracted to src/slot_state.rs as a standalone unit (StateWord:
publish_queued / try_claim / yield_return / park_return / unpark / set_done /
reclaim, plus the Status view for cold paths). runtime.rs keeps the protocol
rationale and consumes the mechanism; every transition self-asserts its
precondition per the assert-the-invariants rule.
src/sync_shim.rs: std vs loom indirection (atomics + UnsafeCell with the
with/with_mut access API) for the two loom-modeled modules. Loom models run
the PRODUCTION code, not a replica:
- slot_state: no-lost-wakeup (park vs unpark), two-unparkers-one-enqueue,
stale-unpark-never-hits-reused-slot (the ABA theorem), unpark-vs-claim.
- run_queue: mpmc exactly-once through a lap wraparound, push/pop race,
striped two-producer drain.
RUSTFLAGS="--cfg loom" cargo test --lib --release — 7 models, all pass.
RawMutex deliberately not loom-modeled (futexes can't be; textbook mutex3
with stress + unwind coverage).
Assertion sweep (invariants now checked at the point of reliance):
- enqueue debug-asserts the word reads EXACTLY (gen, Queued) — the
at-most-once-enqueued invariant the ring capacity proof leans on.
- RawMutex enforces the leaf rule mechanically: debug-build thread-local
held-count, panics at the acquisition that violates it.
- live_actors underflow (double finalize) asserted.
- StateWord transition preconditions asserted (yield/park/done/reclaim/
publish/claim).
Audit: with_runtime, RawMutex guards, run-queue ops, and trace::record all
gate preemption (and thereby the stop sentinel) for their span; trace was
already self-gating. Sole remaining exception is the channel std MutexGuard
— the documented first post-v0.5 fast-follow.
Review fixes to the branched-in work:
- slot_state::status_for mapped (matching gen, Vacant) to Stale instead of
unreachable!: Pid::new is public, so a forged/never-issued pid (e.g.
monitor(Pid::new(5,0)) on a fresh slab) could reach it — "no such actor"
is the correct total answer; issued pids still can't get there.
- Tightened enqueue's assert from Status::Live to the exact (gen, Queued)
word its own comment argues for.
Validated: 22 suites in debug (all asserts + leaf counter live) for
rq-mutex/rq-mpmc/rq-striped, release for rq-mutex, smarm-trace build +
stress, loom 7/7.
- benches/rq_micro.rs: raw-structure microbench, threads x producer:consumer
ratio sweep. Benches all three queue types in one binary (they compile in
every build; only the runtime alias is feature-selected), so no rebuild
dance. Queues sized to the op count so the occupancy contract is met.
- benches/rq_runtime.rs: whole-runtime benches with the selected variant:
yield-storm (pure queue churn), ping-pong-pairs (park/unpark latency),
spawn-storm (slab + free list + queue under churn). Scheduler-count sweep.
- scripts/bench_rq.sh: rebuilds rq_runtime per rq-* feature, runs rq_micro
once, aggregates RQCSV lines into bench_results/summary.csv.
- All knobs via SMARM_BENCH_* env vars; house table format + machine lines.
- run_queue module is now #[doc(hidden)] pub (types + push/pop/len +
MpmcRing::with_capacity) solely so the external bench binary can drive the
raw structures.
docs(roadmap): phase 4 ticked (harness done; numbers from the 20-core box).
New fast-follow per review: assert the invariants we lean on — debug_assert!
on hot paths, loud assert!/panic on cold ones, at the point of reliance;
sweep existing code during the phase-5 audit, adopt as house style.
Validated end-to-end at smoke scale on the 1-core sandbox: full driver run,
24-row summary.csv across micro (3 structures x sweeps) and runtime
(3 variants x 3 benches x thread sweep).
src/run_queue.rs: the run queue extracted behind a compile-time-selected
type alias; mutually-exclusive cargo features with compile_error! guards
(zero or >1 selected). No runtime dispatch. All variants compile in every
build so their unit tests always run; the feature only picks the alias.
- rq-mutex (default): Mutex<VecDeque>, the control/baseline.
- rq-mpmc: hand-rolled Vyukov bounded MPMC ring, per-cell sequence numbers,
padded enqueue/dequeue counters. Strict FIFO, lock-free.
- rq-striped: M Vyukov rings (M = thread_count rounded up to pow2),
fetch-add ticket distribution, probe-from-home on push. Relaxed FIFO with
stripe-bounded skew; Σcapacity ≈ 2×max_actors so the probe terminates.
Capacity soundness: occupancy ≤ max_actors by the at-most-once-enqueued
invariant + slab cap, rings sized ≥ that bound, so push is infallible; a
full ring panics as a double-enqueue invariant violation rather than spin.
Two contracts the extraction made explicit (documented + debug-asserted):
- Queue ops require preemption disabled: a producer suspended between
claiming a cell and publishing its sequence stalls every consumer behind
it — livelock, since the suspended actor's own resume entry is behind the
hole. Structurally guaranteed since the phase-2 with_runtime NoPreempt fix.
- pop()==None is a snapshot, not a fence. Termination is counter-first:
every queue entry's target stays Queued (hence live) until that entry is
popped, so live==0 alone implies nothing actionable is or can be queued;
argument rewritten at the schedule_loop site. SharedState and with_shared
are deleted — nothing global is mutex-guarded on the run path under the
ring variants.
Validated: 22 suites green per variant (release for all three; debug with
live asserts for all three), ring unit tests (FIFO, lap wraparound,
4p/4c exactly-once, skewed drain) in every build, both compile_error!
guards verified to fire.
The slot table split (ROADMAP_v0.5 phase 2): slot lookup is lock-free, the
run path (yield/park/unpark/pop/resume) takes zero locks beyond the queue
mutex itself, and SharedState shrinks to { run_queue }.
Core pieces:
- src/raw_mutex.rs: hand-rolled 3-state futex mutex (Drepper mutex3),
non-poisoning; the guard enters NoPreempt, which both bounds hold times and
structurally closes the unwind-under-lock hole (the stop sentinel shares
the gate). Used for per-slot cold data, the free list, and the stack pool.
- Fixed slab Box<[Slot]> (default max_actors = 16_384, ~4 MiB, align(128)
against false sharing). Slots never move -> stable addresses, lock-free
index. Exhaustion panics loudly, naming Config::max_actors(n) as the fix.
Unbounded/segmented slab stays deferred (see ROADMAP).
- Per-slot AtomicU64 packing (generation << 32 | state), states
Vacant/Queued/Running/RunningNotified/Parked/Done. Every transition CASes
the packed word, so the generation check is atomic with the transition: no
ABA, no spurious unparks on recycled slots. RunningNotified replaces the
pending_unpark bool (the lost-wakeup window is a state, not a flag) and
uniformly fixes a LATENT LOST WAKEUP in the old Blocking-IO completion
path, which set the result for a still-Running actor without flagging it.
- Invariant: a pid is in the run queue at most once (pushes pair 1:1 with
transitions into Queued; only the scheduler does Queued->Running). This is
what makes phase 3's bounded rings sound.
- sp -> relaxed AtomicUsize; stop flag + first-resume closure as AtomicPtrs
(closure double-boxed for a thin pointer, swap-to-take): the resume path is
fully atomic.
- finalize_actor: Done published under the dying slot's cold lock (join's
check-or-register is linearized by it); link cascade locks peers ONE AT A
TIME (cold locks are leaves) with the acyclicity argument written at the
site; link() registers on the target first, then self (stale self-entries
are benign, every walk re-verifies the peer's word).
- Termination by counters: live_actors incremented in spawn pre-enqueue,
decremented at the very END of finalize after all wakeup enqueues; exit on
io_out == 0 (read before the queue lock, phase-1 ordering) && queue empty
&& live == 0. Soundness note at the site: any enqueue targets a live actor.
- spawn boxes the closure and acquires the stack BEFORE any runtime lock: no
allocation ever happens under a global lock anymore.
- with_runtime/try_with_runtime now enter NoPreempt for their full span.
This fixes a bug the rework exposed: install_actor allocated with
preemption enabled while the RUNTIME RefCell borrow was live; a timeslice
preemption there migrates the actor across OS threads and the borrow guard
increments one thread's RefCell count and decrements another's — underflow
to 'permanently mutably borrowed', cascading panics (caught by stress
suite: deterministic non-unwinding-panic abort in lost_wakeup_many_pairs).
The old code was safe only by accident; now it's structural.
- Behavior note: request_stop on a RUNNING target now marks it
RunningNotified, so its next park returns immediately to an observation
point — faster stop observation; parked/queued/done semantics unchanged.
Validated: 22 suites green in release (1/2/8-thread oversubscribed on the
1-core sandbox) and in debug with all debug_asserts live; stress suite x5.
The phase-1 split filled Idle::next_deadline from the timers lock after the
shared lock was released, which turned the stale-PID branch (formerly a 100µs
poll) into a potential sleep-until-timer-deadline while runnable actors sat in
the queue — a hard stall on exact(1). Stale pops are now PopResult::Retry and
loop without sleeping.
test(poison): strengthen regression coverage. The stop-storm test never fired
the sentinel under a lock (its actors only allocate at lock-free points). Add
self_stop_during_spawn_does_not_poison_shared_mutex: a stop-flagged actor whose
next allocation is the Box::new(closure) inside spawn's with_shared. Validated
both ways: passes with the gate, SIGABRTs (unwind-in-allocator) with the gate
removed. Also alloc_interval(1) so every allocation is an observation point.
- next_monitor_id -> AtomicU64 on RuntimeInner
- timers -> own Mutex<Timers>; io -> own Mutex<Option<IoThread>> (lock order: io-before-shared)
- pending_closures Vec folded into Slot::pending_closure
- termination check reads io liveness before shared; ordering argument documented
- poison fix: check_cancelled() no longer fires while preemption is disabled,
so a cancellation unwind can never poison a runtime/channel mutex
- regression test: tests/poison_stop.rs
- ROADMAP_v0.5.md added
Mainline now carries roadmap #1-#5; the aarch64 context-switch port is a single untested commit on the arm-port branch. Update the task.md resume block and the README build section accordingly.
Monitors could be installed but never taken down. That gap was about to
bite: the gen_server call timeout we want next is the Erlang dance —
monitor the server, wait for the reply or a Down or a deadline, then
demonitor — and without a way to remove a registration, every timed-out
call would leak a monitor on the server's slot and risk a stale Down
arriving later. So this lands the cleanup primitive before anything
depends on the old shape.
The decision flagged in the roadmap ("decide the monitor API NOW") is
resolved by giving each registration a process-unique MonitorId and
returning it to the caller. monitor() now hands back a
Monitor { id, target, rx } rather than a bare Receiver<Down>: read the
notice from rx as before, and pass &Monitor to demonitor to tear exactly
that registration down. The id comes from a monotonic counter in shared
state, bumped under the same lock that does the registration, so it's
deterministic and never reused — which is what lets demonitor name one of
several monitors on the same target unambiguously. target rode along on
the struct (over the sketched {id, rx}) purely so demonitor can go
straight to the slot instead of scanning every slot for the id.
demonitor returns Option<MonitorId> rather than a bool: Some(id) when a
live registration was found and removed, None when there was nothing left
to remove — it already fired (the registration is drained on finalize),
it was a NoProc, or the slot has been reclaimed. The generation half of
the pid quietly protects that last case: a recycled slot index fails
slot_mut's generation check, so a late demonitor is a clean no-op and can
never strip a different actor's monitor that happens to share the index.
The one piece of real care is reentrancy. Removing a registration drops
the slot's Sender, and Sender::drop can unpark a parked receiver, which
re-enters the shared mutex — which is not reentrant. So demonitor moves
the sender out of the Vec under the lock and lets it drop only after the
lock is released, the same discipline finalize_actor already follows for
its monitor and supervisor sends.
Flushing a Down the target already queued isn't a separate flag; it falls
out of dropping the Monitor. demonitor(&m); drop(m) stops future notices
and discards any queued one — the gen_server-call cleanup in one move.
Storage is now Vec<(MonitorId, Sender<Down>)>. The three slot-reset sites
were left alone on purpose: they clear/rebuild the Vec, which doesn't care
about the element type, so there's no fourth reset obligation. finalize
just destructures (_, m). Because chained_spawn and yield_many register no
monitors, that Vec stays empty on the hot path — taking an empty Vec costs
the same and the notify loop runs zero times — and a before/after general
probe confirmed both medians sit within noise.
Tests cover the three behaviours that matter: a demonitored watcher gets
no Down (its channel closes), demonitoring one of several leaves the
siblings firing normally, and demonitoring after the Down has already
fired reports None.
smarm-agent