FdArm composes fd readiness with channel arms on one wait epoch.
Selectable grows fallible sel_register and an eager-cleanup hook;
losing/stop-unwound/timed-out fd arms are unregistered (waiters entry
+ kernel ONESHOT) so the fd is never poisoned. try_select /
try_select_timeout surface registration errors (EBADF, EMFILE,
AlreadyExists) instead of RFC 008's permanently-ready lean, which
would busy-loop a healthy-but-unregistrable fd; select/select_timeout
stay infallible for channel-only arms. Adds wait_readable_timeout /
wait_writable_timeout as one-arm selects.
Known benign race (pre-existing, slightly widened): a queued FdReady
racing the cleanup DEL can spuriously wake a fresh waiter on that fd;
absorbed by select's defensive re-loop. Fixable by epoch-stamping
completions.
Shootout (6d9f369 harness, 24-core sweep) landed in RFC 005's World 3:
queue variants indistinguishable in rq_runtime; per-wake latency is the
ceiling. Record the queue-topology closure (rq-mutex frozen as default,
benchmark-driven reopening only), define v0.9 as the wake-path latency
cycle (RFC 005 slot -> bench -> RFC 004 spinning workers -> bench +
interaction pass), and add per-switch cost to Later pending a profiling
spike + RFC.
Completed cycles compacted to the two trailing minors (v0.7, v0.8);
earlier history lives in git.
- Replace v0.3 content throughout; bump version label to v0.8
- New SVG: Krebs-cycle scheduler loop (6 stations, actor handoff as
energy exchange, finalize leaving the cycle)
- New SVG: gen_server dual-plane (model vs implementation mapping)
- New SVG: slot state machine with packed word, park-epoch and all
transitions (loom theorems annotated)
- Updated SVG: module map (22 modules, 4 layers incl. slot_state,
run_queue, raw_mutex, sync_shim, monitor, link, registry, trace)
- Updated SVG: dep graph (runtime hub, OTP layer on public surface)
- New sections: Slot State & Park-Epoch, Run Queue variants,
GenServer, OTP (monitors/links/supervisors/registry)
- Updated: init (slab allocation, counter-based termination), spawn
(closure in slot AtomicPtr, stack pool, live_actors ordering),
preemption (stop sentinel + StopSentinel/Outcome::Stopped),
IO (epoch-matched completions, fd-leak fix), gotchas (lost-wakeup
and global-mutex flipped to green; 4 new cards)
- Yield-sources table grown to 9 rows (select, recv_timeout,
call, request_stop)
- Fix: swap local css2.css font ref for Google Fonts CDN
A stopped actor unwinding out of wait_fd's park leaked its waiters entry
and the kernel-side EPOLLONESHOT registration; the stale entry then failed
every future wait_*() on that fd with AlreadyExists (the defensive bare DEL
in epoll_register sits behind the contains_key check, so it never ran).
Fix where the invariant breaks: a drop guard in wait_fd, armed after a
successful register and forgotten on the normal wake (where FdReady already
removed + DEL'd). On unwind it cleans up iff the entry is still this wait's
(pid, epoch) — an entry consumed by a racing FdReady means the fd may carry
another actor's fresh registration, which must be left alone. Closes the
v0.2 fd-hygiene TODO.
Monitors are created at runtime (you watch a worker you just spawned in a
handler), so down arms can't ride the static info list. init grows a
&ServerCtx parameter (breaking; no-op default) whose clonable Watcher hands
Monitors to the loop over a control arm; the loop selects the live down
arms ahead of everything else. Arm priority: downs → control → infos →
inbox. A delivered Down retires its arm (monitors are one-shot); a state
that never clones the Watcher closes the control arm after init and the
loop falls back to the plain-inbox park.
type Info + handle_info (no-op default) on GenServer; ServerBuilder
(with_info/under/start) so start variants don't multiply — start/start_under
stay as wrappers. The loop selects [infos.., inbox] in priority order when
info arms exist, and keeps the plain recv() park when none do. Closed info
arms are silently removed (closed-arm-is-ready-forever); a closed inbox
still means graceful shutdown.
Gotchas now lead with the consuming-wake invariant ('the only wildcard wake
is request_stop') and the obligations it puts on future wakers. Deviations
recorded: the no-park exit's retire_wait/clear_notify (a protocol piece the
plan missed), Drop-guard deregistration superseded by own-pid assert
relaxation, and closed arms being ready forever.
The deadline is one more stamped waker on the select's wait epoch: a unit
TimerTarget whose on_timeout is a bare unpark_at. Nothing registers in any
arm for it, so there is nothing to cancel or leak — an arm winning leaves
the timer entry to die at its epoch CAS; the timer winning leaves the arms'
registrations to self-clean like any select loser's. Classification is pure
channel state (wakes are precise): some arm ready -> Some(first, priority
order); none -> the timer was the only remaining stamped waker -> None.
Message-first on a raced deadline, same as recv_timeout.
Registration pass factored into register_arms, which owns the retire_wait
obligation on the ready-now exit for both entry points.
select(&[&dyn Selectable]) -> usize registers (pid, epoch) in every arm
under one wait epoch, parks once, and returns the first ready index in
documented priority order (BEAM-style; no fairness promise). A closed arm
counts as ready — and stays ready forever, so callers drop it from the set
once observed. Losing arms need no cancellation pass: the winning wake
consumed the epoch, so their registrations die at their wakers' failed CAS
or are overwritten by the receiver's next wait on that channel (the
single-receiver debug_asserts relax to 'none or own pid' accordingly).
The one genuinely new protocol piece is the no-park exit: returning with an
arm ready at registration time leaves earlier arms holding LIVE-epoch
registrations, whose wakes could fault a later one-shot park as a pending
notification. scheduler::retire_wait closes it — bump the epoch (in-flight
wakes die at their CAS), eat a notification that already landed
(StateWord::clear_notify), then re-observe the stop flag, in that order.
Proved by the new loom theorem retire_eats_late_arm_notification; the
integration probes in tests/select.rs fire stale loser-arm wakes and assert
a subsequent sleep's one-shot park holds its full duration.
The slot epoch is THE wait identity, so the hand-rolled per-primitive copies
go away: channel loses cur_wait/next_wait_seq/timed_out, mutex loses Wait.seq
and next_seq, TimerTarget::on_timeout takes the epoch. Registrations become
(pid, epoch) — channel parked_receiver, mutex waiters, io fd waiters,
Blocking io completions, sleep timers, joiner lists — and their wakers move
to unpark_at. begin_wait is lock-free, so each primitive opens the wait
inside the same critical section that publishes the registration.
recv_timeout's wake-classification loop collapses: wakes are precise, so
queue → Ok, senders==0 → Disconnected, else Timeout — the 'Defensive'
re-register branch is now unreachable by protocol, not by audit. Same for
Mutex::lock_timeout's one-shot park.
End-state invariant, auditable in one sentence: the only wildcard wake is
request_stop, which is terminal.
Word repacks to (gen << 32) | (epoch << 8) | state. begin_wait opens a wait
identity; every successful wake consumes it, so at most one wake lands per
wait by construction. unpark grows an Option<epoch> match; RuntimeInner gains
unpark_at. Claim/yield/park transitions become epoch-preserving CAS loops.
Loom: all four prior theorems re-proved on the new word, plus
consumed_epoch_unpark_never_lands — a late waker stamped with a consumed
epoch can neither enqueue nor notify, in every interleaving (the property
select's loser arms and every one-shot park site lean on).
No callers stamped yet; behaviour identical. Two transient dead_code
warnings (begin_wait, unpark_at) are consumed by the next commit.
Marks the three v0.6 items done with their hashes and the load-bearing
deviations: two-class lock order replacing the strict leaf rule, the
registry as a bimap with name_of and without send_by_name (no per-pid
mailbox to send to), and call_timeout built on recv_timeout instead of
monitor machinery. Adds the Leaf -> Channel order to the standing
invariants; README module table picks up recv_timeout, call_timeout,
and the registry row.
Deviates deliberately from the roadmap's monitor-based sketch
(monitor + wait reply-or-Down-or-deadline + demonitor): server death is
already observable on the reply channel itself - the reply sender drops
as the server unwinds, closing the channel and waking the parked caller.
So a bounded call is exactly recv_timeout on the reply channel, mapping
Disconnected -> ServerDown and Timeout -> Timeout. No registration
exists, so a timed-out call leaks nothing by construction - the
guarantee the monitor design had to engineer.
Semantics on timeout match Erlang: the request stays in the inbox and
is still handled; the late reply's send fails harmlessly against the
dropped reply receiver. CallTimeoutError keeps ServerDown and Timeout
distinguishable; the infallible call() is unchanged.
Tests: reply within deadline, timeout against a slow (parking) handler
with elapsed bounds, server survival across an abandoned call (late
reply discarded, bounded and unbounded calls keep working), and
ServerDown-not-Timeout for both death paths (mid-call panic and
already-gone inbox).
The per-recv deadline the roadmap deferred, built exactly the way
Mutex::lock_timeout already works: register the wait with a per-wait
seq, arm a timer::Reason::WaitTimeout with the channel inner (now a
TimerTarget) as the target, park. On expiry on_timeout cancels the wait
only if that same seq is still parked; satisfied or abandoned waits
leave a stale heap entry that no-ops on seq mismatch, per the
no-cancellation convention in timer.rs.
Race resolution is message-first: a send that lands by the time the
woken receiver runs is delivered even if the deadline had also passed.
Closure is reported as RecvTimeoutError::Disconnected, keeping timeout
and server/sender death distinguishable for callers (gen_server call
timeout builds on exactly this distinction next).
The timer is armed outside the channel critical section (the timers
lock must never nest under a Channel-class lock); the unpark race this
opens is absorbed by the RunningNotified protocol.
Tests: immediate delivery, actual timeout (with elapsed check), prompt
wake on send, Disconnected on close, zero-duration poll, post-timeout
seq isolation (stale entry must not cancel later waits), and a
4-scheduler mixed-outcome run (12 fed / 12 timed out).
register/whereis/unregister plus the inverse lookup name_of. Erlang
semantics: one name per pid, one pid per name, registering over a live
binding errors; same-binding re-register is an idempotent Ok.
Bimap = two HashMaps held in exact inverse under one Leaf RawMutex in
RuntimeInner; the invariant is debug-asserted on every mutation. The
inverse direction costs one extra String per binding and buys O(1)
pid->name for supervisors/tracing/diagnostics.
Cleanup is lazy: no finalize_actor hook, no Slot field (which would buy
into the reset-in-three-places invariant). Liveness rides the
generation-checked slot word - pids are never reused, so a stale binding
is detectable, never misdirected - and bindings to dead actors behave as
absent, pruned on contact by whereis/name_of/register/unregister.
Liveness checks under the registry lock read only the atomic slot word,
so the leaf rule holds trivially.
send_by_name is deliberately absent: smarm has no per-pid mailbox, so
there is nothing generic to send *to* - the registry yields a Pid,
usable with monitor/link/request_stop and as routing for user channels.
Tests: roundtrip both directions, idempotence, NameTaken on live holder,
one-name-per-pid, NoProc, death-evaporates-binding (via lookup pruning
and via register's own eviction path), unregister, cross-actor lookup
wired into request_stop, and a 4-scheduler name-contention churn test.
Phase 2 surfaced the hole this closes: channel guards were held with
preemption enabled, so a timeslice switch inside a channel critical
section could release the pthread mutex from a different OS thread (UB).
RawMutex disables preemption for the guard span and is
cross-thread-release sound by construction; poison goes away with it.
The strict single-class leaf rule cannot survive the migration: finalize
clones the supervisor/trap senders and monitor() clones the Down sender,
all under a cold lock, and those senders live in the slot - the nesting
is structural. The leaf check is therefore generalized to two classes:
Leaf (cold locks, free list, stack pool) and Channel. Order is
Leaf -> Channel, at most one of each; both directions of violation are
debug-asserted at the acquisition site. Channel critical sections call
only the lock-free unpark protocol, so the order is acyclic.
Tests: class-ordering unit tests in raw_mutex (allowed nesting + all
three rejected shapes), plus a multi-scheduler integration test driving
channels through monitor churn and actor death so any ordering
regression trips the debug assert instead of deadlocking.
benches/README.md: catalog of the two bench families (cross-runtime
comparisons + the v0.5 run-queue shootout), how to run the shootout
(scripts/bench_rq.sh, SMARM_BENCH_* knobs, the --no-default-features dance,
summary.csv/RQCSV format), honest-numbers caveats (core count is the
experiment; striped losing at low thread counts is expected), and the
conventions for adding a bench.
tests/README.md: catalog grouped by layer (low-level units / feature areas /
regression+stress), the run matrix (debug-first — that's where the invariant
asserts live — three queue variants, release, loom, trace), why loom tests
live in src/ rather than tests/, and the house conventions: one runtime per
test, oversubscription on purpose, regression tests validated against the
reintroduced bug, odds-stacking for stochastic tests, ordering-not-duration
time assertions, and new-invariant = assert + loom model.
Also fixes a stale header in tests/mutex.rs that claimed 'loom::Mutex' —
it tests smarm::Mutex, and the old name is actively confusing now that loom
is real in this repo.
State machine extracted to src/slot_state.rs as a standalone unit (StateWord:
publish_queued / try_claim / yield_return / park_return / unpark / set_done /
reclaim, plus the Status view for cold paths). runtime.rs keeps the protocol
rationale and consumes the mechanism; every transition self-asserts its
precondition per the assert-the-invariants rule.
src/sync_shim.rs: std vs loom indirection (atomics + UnsafeCell with the
with/with_mut access API) for the two loom-modeled modules. Loom models run
the PRODUCTION code, not a replica:
- slot_state: no-lost-wakeup (park vs unpark), two-unparkers-one-enqueue,
stale-unpark-never-hits-reused-slot (the ABA theorem), unpark-vs-claim.
- run_queue: mpmc exactly-once through a lap wraparound, push/pop race,
striped two-producer drain.
RUSTFLAGS="--cfg loom" cargo test --lib --release — 7 models, all pass.
RawMutex deliberately not loom-modeled (futexes can't be; textbook mutex3
with stress + unwind coverage).
Assertion sweep (invariants now checked at the point of reliance):
- enqueue debug-asserts the word reads EXACTLY (gen, Queued) — the
at-most-once-enqueued invariant the ring capacity proof leans on.
- RawMutex enforces the leaf rule mechanically: debug-build thread-local
held-count, panics at the acquisition that violates it.
- live_actors underflow (double finalize) asserted.
- StateWord transition preconditions asserted (yield/park/done/reclaim/
publish/claim).
Audit: with_runtime, RawMutex guards, run-queue ops, and trace::record all
gate preemption (and thereby the stop sentinel) for their span; trace was
already self-gating. Sole remaining exception is the channel std MutexGuard
— the documented first post-v0.5 fast-follow.
Review fixes to the branched-in work:
- slot_state::status_for mapped (matching gen, Vacant) to Stale instead of
unreachable!: Pid::new is public, so a forged/never-issued pid (e.g.
monitor(Pid::new(5,0)) on a fresh slab) could reach it — "no such actor"
is the correct total answer; issued pids still can't get there.
- Tightened enqueue's assert from Status::Live to the exact (gen, Queued)
word its own comment argues for.
Validated: 22 suites in debug (all asserts + leaf counter live) for
rq-mutex/rq-mpmc/rq-striped, release for rq-mutex, smarm-trace build +
stress, loom 7/7.
- benches/rq_micro.rs: raw-structure microbench, threads x producer:consumer
ratio sweep. Benches all three queue types in one binary (they compile in
every build; only the runtime alias is feature-selected), so no rebuild
dance. Queues sized to the op count so the occupancy contract is met.
- benches/rq_runtime.rs: whole-runtime benches with the selected variant:
yield-storm (pure queue churn), ping-pong-pairs (park/unpark latency),
spawn-storm (slab + free list + queue under churn). Scheduler-count sweep.
- scripts/bench_rq.sh: rebuilds rq_runtime per rq-* feature, runs rq_micro
once, aggregates RQCSV lines into bench_results/summary.csv.
- All knobs via SMARM_BENCH_* env vars; house table format + machine lines.
- run_queue module is now #[doc(hidden)] pub (types + push/pop/len +
MpmcRing::with_capacity) solely so the external bench binary can drive the
raw structures.
docs(roadmap): phase 4 ticked (harness done; numbers from the 20-core box).
New fast-follow per review: assert the invariants we lean on — debug_assert!
on hot paths, loud assert!/panic on cold ones, at the point of reliance;
sweep existing code during the phase-5 audit, adopt as house style.
Validated end-to-end at smoke scale on the 1-core sandbox: full driver run,
24-row summary.csv across micro (3 structures x sweeps) and runtime
(3 variants x 3 benches x thread sweep).
src/run_queue.rs: the run queue extracted behind a compile-time-selected
type alias; mutually-exclusive cargo features with compile_error! guards
(zero or >1 selected). No runtime dispatch. All variants compile in every
build so their unit tests always run; the feature only picks the alias.
- rq-mutex (default): Mutex<VecDeque>, the control/baseline.
- rq-mpmc: hand-rolled Vyukov bounded MPMC ring, per-cell sequence numbers,
padded enqueue/dequeue counters. Strict FIFO, lock-free.
- rq-striped: M Vyukov rings (M = thread_count rounded up to pow2),
fetch-add ticket distribution, probe-from-home on push. Relaxed FIFO with
stripe-bounded skew; Σcapacity ≈ 2×max_actors so the probe terminates.
Capacity soundness: occupancy ≤ max_actors by the at-most-once-enqueued
invariant + slab cap, rings sized ≥ that bound, so push is infallible; a
full ring panics as a double-enqueue invariant violation rather than spin.
Two contracts the extraction made explicit (documented + debug-asserted):
- Queue ops require preemption disabled: a producer suspended between
claiming a cell and publishing its sequence stalls every consumer behind
it — livelock, since the suspended actor's own resume entry is behind the
hole. Structurally guaranteed since the phase-2 with_runtime NoPreempt fix.
- pop()==None is a snapshot, not a fence. Termination is counter-first:
every queue entry's target stays Queued (hence live) until that entry is
popped, so live==0 alone implies nothing actionable is or can be queued;
argument rewritten at the schedule_loop site. SharedState and with_shared
are deleted — nothing global is mutex-guarded on the run path under the
ring variants.
Validated: 22 suites green per variant (release for all three; debug with
live asserts for all three), ring unit tests (FIFO, lap wraparound,
4p/4c exactly-once, skewed drain) in every build, both compile_error!
guards verified to fire.
The slot table split (ROADMAP_v0.5 phase 2): slot lookup is lock-free, the
run path (yield/park/unpark/pop/resume) takes zero locks beyond the queue
mutex itself, and SharedState shrinks to { run_queue }.
Core pieces:
- src/raw_mutex.rs: hand-rolled 3-state futex mutex (Drepper mutex3),
non-poisoning; the guard enters NoPreempt, which both bounds hold times and
structurally closes the unwind-under-lock hole (the stop sentinel shares
the gate). Used for per-slot cold data, the free list, and the stack pool.
- Fixed slab Box<[Slot]> (default max_actors = 16_384, ~4 MiB, align(128)
against false sharing). Slots never move -> stable addresses, lock-free
index. Exhaustion panics loudly, naming Config::max_actors(n) as the fix.
Unbounded/segmented slab stays deferred (see ROADMAP).
- Per-slot AtomicU64 packing (generation << 32 | state), states
Vacant/Queued/Running/RunningNotified/Parked/Done. Every transition CASes
the packed word, so the generation check is atomic with the transition: no
ABA, no spurious unparks on recycled slots. RunningNotified replaces the
pending_unpark bool (the lost-wakeup window is a state, not a flag) and
uniformly fixes a LATENT LOST WAKEUP in the old Blocking-IO completion
path, which set the result for a still-Running actor without flagging it.
- Invariant: a pid is in the run queue at most once (pushes pair 1:1 with
transitions into Queued; only the scheduler does Queued->Running). This is
what makes phase 3's bounded rings sound.
- sp -> relaxed AtomicUsize; stop flag + first-resume closure as AtomicPtrs
(closure double-boxed for a thin pointer, swap-to-take): the resume path is
fully atomic.
- finalize_actor: Done published under the dying slot's cold lock (join's
check-or-register is linearized by it); link cascade locks peers ONE AT A
TIME (cold locks are leaves) with the acyclicity argument written at the
site; link() registers on the target first, then self (stale self-entries
are benign, every walk re-verifies the peer's word).
- Termination by counters: live_actors incremented in spawn pre-enqueue,
decremented at the very END of finalize after all wakeup enqueues; exit on
io_out == 0 (read before the queue lock, phase-1 ordering) && queue empty
&& live == 0. Soundness note at the site: any enqueue targets a live actor.
- spawn boxes the closure and acquires the stack BEFORE any runtime lock: no
allocation ever happens under a global lock anymore.
- with_runtime/try_with_runtime now enter NoPreempt for their full span.
This fixes a bug the rework exposed: install_actor allocated with
preemption enabled while the RUNTIME RefCell borrow was live; a timeslice
preemption there migrates the actor across OS threads and the borrow guard
increments one thread's RefCell count and decrements another's — underflow
to 'permanently mutably borrowed', cascading panics (caught by stress
suite: deterministic non-unwinding-panic abort in lost_wakeup_many_pairs).
The old code was safe only by accident; now it's structural.
- Behavior note: request_stop on a RUNNING target now marks it
RunningNotified, so its next park returns immediately to an observation
point — faster stop observation; parked/queued/done semantics unchanged.
Validated: 22 suites green in release (1/2/8-thread oversubscribed on the
1-core sandbox) and in debug with all debug_asserts live; stress suite x5.
The phase-1 split filled Idle::next_deadline from the timers lock after the
shared lock was released, which turned the stale-PID branch (formerly a 100µs
poll) into a potential sleep-until-timer-deadline while runnable actors sat in
the queue — a hard stall on exact(1). Stale pops are now PopResult::Retry and
loop without sleeping.
test(poison): strengthen regression coverage. The stop-storm test never fired
the sentinel under a lock (its actors only allocate at lock-free points). Add
self_stop_during_spawn_does_not_poison_shared_mutex: a stop-flagged actor whose
next allocation is the Box::new(closure) inside spawn's with_shared. Validated
both ways: passes with the gate, SIGABRTs (unwind-in-allocator) with the gate
removed. Also alloc_interval(1) so every allocation is an observation point.
- next_monitor_id -> AtomicU64 on RuntimeInner
- timers -> own Mutex<Timers>; io -> own Mutex<Option<IoThread>> (lock order: io-before-shared)
- pending_closures Vec folded into Slot::pending_closure
- termination check reads io liveness before shared; ordering argument documented
- poison fix: check_cancelled() no longer fires while preemption is disabled,
so a cancellation unwind can never poison a runtime/channel mutex
- regression test: tests/poison_stop.rs
- ROADMAP_v0.5.md added
Mainline now carries roadmap #1-#5; the aarch64 context-switch port is a single untested commit on the arm-port branch. Update the task.md resume block and the README build section accordingly.
Monitors could be installed but never taken down. That gap was about to
bite: the gen_server call timeout we want next is the Erlang dance —
monitor the server, wait for the reply or a Down or a deadline, then
demonitor — and without a way to remove a registration, every timed-out
call would leak a monitor on the server's slot and risk a stale Down
arriving later. So this lands the cleanup primitive before anything
depends on the old shape.
The decision flagged in the roadmap ("decide the monitor API NOW") is
resolved by giving each registration a process-unique MonitorId and
returning it to the caller. monitor() now hands back a
Monitor { id, target, rx } rather than a bare Receiver<Down>: read the
notice from rx as before, and pass &Monitor to demonitor to tear exactly
that registration down. The id comes from a monotonic counter in shared
state, bumped under the same lock that does the registration, so it's
deterministic and never reused — which is what lets demonitor name one of
several monitors on the same target unambiguously. target rode along on
the struct (over the sketched {id, rx}) purely so demonitor can go
straight to the slot instead of scanning every slot for the id.
demonitor returns Option<MonitorId> rather than a bool: Some(id) when a
live registration was found and removed, None when there was nothing left
to remove — it already fired (the registration is drained on finalize),
it was a NoProc, or the slot has been reclaimed. The generation half of
the pid quietly protects that last case: a recycled slot index fails
slot_mut's generation check, so a late demonitor is a clean no-op and can
never strip a different actor's monitor that happens to share the index.
The one piece of real care is reentrancy. Removing a registration drops
the slot's Sender, and Sender::drop can unpark a parked receiver, which
re-enters the shared mutex — which is not reentrant. So demonitor moves
the sender out of the Vec under the lock and lets it drop only after the
lock is released, the same discipline finalize_actor already follows for
its monitor and supervisor sends.
Flushing a Down the target already queued isn't a separate flag; it falls
out of dropping the Monitor. demonitor(&m); drop(m) stops future notices
and discards any queued one — the gen_server-call cleanup in one move.
Storage is now Vec<(MonitorId, Sender<Down>)>. The three slot-reset sites
were left alone on purpose: they clear/rebuild the Vec, which doesn't care
about the element type, so there's no fourth reset obligation. finalize
just destructures (_, m). Because chained_spawn and yield_many register no
monitors, that Vec stays empty on the hot path — taking an empty Vec costs
the same and the notify loop runs zero times — and a before/after general
probe confirmed both medians sit within noise.
Tests cover the three behaviours that matter: a demonitored watcher gets
no Down (its channel closes), demonitoring one of several leaves the
siblings firing normally, and demonitoring after the Down has already
fired reports None.
Thin request-reply layer on channels, no runtime change. A server is an
actor owning a GenServer state; clients hold a clonable ServerRef and issue
call (sync, parks for the reply) or cast (fire-and-forget).
- Single inbox carrying an internal Envelope { Call(req, reply_tx) | Cast },
forced by the no-select / no-unified-mailbox invariant; call makes a
one-shot reply channel and parks on it.
- Server-down detection is pure channel closure (no monitor): send fails if
the inbox is gone; the reply sender drops on the server's unwind so a
parked caller wakes to Err. Both collapse to CallError/CastError::ServerDown.
- init/terminate are optional trait hooks; terminate runs via a drop guard so
it fires on every exit path (clean close, panic, request_stop). Must stay
non-blocking — may run mid-unwind.
- ServerRef carries pid() for monitor/request_stop/link; start + start_under.
Tests: cast-then-call roundtrip, init/terminate ordering, both server-down
paths (reply-channel close on handler panic; inbox-send failure when gone).
Record what landed for recv_match/try_recv_match and update the
no-select gotcha: selective receive stayed per-channel rather than
introducing a cross-channel mailbox.
recv_match(pred) scans the queue front-to-back, removes and returns the
first match (rest preserved in arrival order), and parks/re-scans on every
send when nothing matches — a selective receiver may park on a non-empty
queue. Returns Err only once the channel is closed with no queued match.
try_recv_match is the non-blocking variant, mirroring try_recv.
Sender::drop now wakes the parked receiver on the last-sender drop
regardless of queue emptiness, so a selective receiver parked on a
non-empty no-match queue observes closure instead of sleeping forever.
No-op for plain recv (which only ever parks on an empty queue).
- task.md: mark roadmap #3 done (record the resolved dedicated-inbox
decision + test list), add the feature commit to the resume block, note
the now-resolved trap_exit channel question, and list spawn_link under #5.
- supervisor.rs: the module comment predated cancellation; rewrite it to say
sibling/link/shutdown stops are cooperative (request_stop), not that
one_for_all / rest_for_one / links are impossible.
- README: add monitor + link rows, refresh the supervisor row and src layout,
and drop restart-intensity caps from "what's not here" (shipped in #2).
Add Erlang-style process links so an abnormal death fate-shares across a
link set, with trap_exit to convert that into a message instead.
- Slot.links: Vec<Pid>, bidirectional; reset in all three lifecycle sites.
- Actor.trap: Option<Sender<ExitSignal>>, fresh per spawn (a restarted
child starts un-trapped; no fourth reset site).
- link/unlink free functions on self_pid(); trap_exit() -> Receiver<ExitSignal>
(a dedicated inbox, distinct from the monitor Down channel).
- finalize_actor clears reverse links under the lock (always; keeps the
cascade acyclic), then propagates abnormal deaths post-lock: trapping peer
gets an ExitSignal message and survives, non-trapping peer is request_stop'd.
Normal exit never propagates. Dead-pid link delivers an immediate NoProc
(message if trapping, else request_stop(self) -- never a silent no-op).
- ExitSignal reuses DownReason and carries no panic payload (joiner-only).
Tests in tests/link.rs cover the propagate/trap/normal/dead-pid/unlink cases.
Capture the handoff roadmap in-repo, with #1 and #2 marked complete
(commit SHAs, decisions taken, and the orphaned-timer bug fixed in
e80334b). #3 (links + trap_exit) flagged as next.
Adds a Strategy enum (OneForOne default, OneForAll, RestForOne) selected
via OneForOne::strategy(). The triggering child's Restart policy still
decides whether anything restarts; the strategy decides which siblings
are cycled with it:
- OneForAll : all live siblings
- RestForOne: siblings started after the failed child
Group restarts stop the affected survivors cooperatively (request_stop)
in reverse start order, await each one's termination signal on the
existing funnel (no new channel, no select), then restart the whole set
in start order. Signals that arrive for a child we aren't currently
stopping are stashed and replayed by the main loop. A Stopped signal now
counts as abnormal for the restart decision.
On giving up (intensity cap) or any exit with survivors, an ordered
shutdown stops the remaining children in reverse start order instead of
leaking them; on the normal all-settled exit it's a no-op.
The struct keeps the OneForOne name for compatibility (existing tests
unchanged); rename is a later refactor.
A cooperatively-cancelled actor that was sleeping (or in a bounded wait)
leaves its timer entry behind in the heap. The scheduler's shutdown
check required "no pending timers", so a single orphaned far-future
deadline kept the whole runtime alive until it fired — e.g. cancelling
an actor mid sleep(30s) hung run() for 30s.
A timer can only ever do useful work by waking a live actor, so once no
actors are live every remaining entry is orphaned by definition. Drop
the pending-timers condition from the shutdown check (live == 0 already
implies nothing a timer could wake) and clear the heap on the way out.
request_stop(pid) sets a per-actor flag and wakes a parked target. The
actor realizes the stop as a controlled unwind at its next observation
point (check!()/alloc via maybe_preempt, or the wakeup side of any
blocking park): a StopSentinel panic tears the stack down via the
trampoline's existing catch_unwind, running Drop, and is reported as the
new Outcome::Stopped (distinct from a user Panic).
Death surface kept distinct from Exit: Signal::Stopped(pid) +
DownReason::Stopped, so the supervisor await logic to come can tell a
requested stop apart from a self-termination.
Flag lives on Actor behind Arc<AtomicBool>; the scheduler resume path
takes a raw pointer into it (no per-resume refcount traffic), keeping
yield throughput at baseline.
Builds on the existing supervisor_channel funnel (one mailbox per
supervisor; better than N monitor channels given there is no select).
- supervisor.rs: Restart{Permanent,Transient,Temporary}, ChildSpec (an
Fn factory so children can be re-instantiated), and OneForOne with a
builder (child/intensity) and run() supervision loop. Restart decision
keys off whether the Signal was a panic; a sliding-window intensity cap
stops crash loops and returns instead of spinning.
- Does NOT forcibly terminate children: shared-heap + Drop make async
teardown of a peer unsound, so one_for_all/rest_for_one and true links
wait on a cooperative-cancellation primitive. Cap-trip just stops
restarting; live children are left as-is.
tests/supervisor.rs: transient restart-then-settle, transient/temporary
no-restart, permanent crash-loop hitting the cap, and one-for-one
isolation across two children. Full suite green.
`monitor(pid) -> Receiver<Down>` lets any actor watch any other and
receive a single Down{pid, reason} when it terminates. Generalizes the
existing single supervisor_channel (which is just a hard-wired monitor
the parent installs at spawn).
- src/monitor.rs: Down / DownReason{Exit,Panic,NoProc} + monitor().
Payload-free: a panic payload has one owner and goes to the joiner via
JoinError, so monitors learn only that it panicked. Monitoring an
already-gone pid yields NoProc immediately.
- runtime: Slot grows `monitors: Vec<Sender<Down>>`; finalize_actor
drains and notifies them outside the shared lock (send can unpark a
receiver, which re-takes the non-reentrant mutex). Cleared in
reclaim_slot and on slot reuse at spawn.
- Registration and finalize both serialize on the shared mutex, so a
target alive at registration always delivers a real Down (no race
between liveness check and registration).
tests/monitor.rs: Exit, Panic, NoProc-on-dead-target, and fan-out to
multiple monitors. Full suite green.
The context switch in context.rs saves no SSE/XMM state, justified by
every yield crossing a Rust `call` boundary (SysV: XMM0-15 caller-saved,
so live XMM is spilled before the call). The non-obvious path is
preemption: check!() inlines maybe_preempt(), so a switch can fire
mid-floating-point-loop rather than at a visible call site.
Verified the assumption holds on that path, and *why*: switch_to_scheduler
is extern "C", so the compiler spills live XMM around it even when the
yield is buried inside an inlined preempt check. Disasm of the probe loop
shows the accumulators spilled to (%rsp) immediately before the preempt
path and reloaded after. Behavioural check: an FP loop preempted mid-flight
produces a bit-identical result to the same workload with no preemption
(abs diff = 0, debug and release). Confirmed non-vacuous under llmdbg:
switch_to_scheduler fires repeatedly during the loop.
This is a confirmation, not a fix; context.rs is unchanged. The test exists
to catch a future regression that reaches the switch without crossing the
extern "C" call boundary (e.g. fully-inlined asm with no call, or async
signal-driven preemption) — either WOULD require saving XMM.
The single-scheduler hot path acquired the shared mutex three times per
yield: once to drain timers, once to drain IO completions, and once to pop
the next runnable actor. It also called Instant::now() every loop iteration
to feed timers.pop_due(), even though the pure-yield / pure-compute workloads
never arm a timer.
Confirmed with llmdbg before touching anything: breakpointing
switch_to_scheduler and instruction-counting one scheduler-side yield cycle on
the ctx_switch_probe showed ~6950 instructions / 68 calls of bookkeeping
between consecutive actor resumes, dominated by lock acquisition and the
per-iteration clock read — not by the naked-asm switch itself (~16 insns).
The te!() trace macro compiles to () without the smarm-trace feature, so it
was ruled out as a contributor.
This collapses the two drain `with_shared` calls into one lock hold and reads
the clock only when the timer heap is non-empty. IO completions are still
drained unconditionally while the IO subsystem is live, because the IO/pool
threads push completions onto their own mutex (not `shared`), so the scheduler
cannot know in advance whether any arrived — it must look; that look is a
single empty-VecDeque check on the hot path. No change to timer or IO
semantics; the asm switch is untouched.
Measured on this box (single core, nproc==1), medians over the bench harness,
re-baselined locally first:
yield_many smarm 1-thread 38459 -> 29943 us (-22.1%)
yield_in_hot_loop smarm 1-thread 166220 -> 122629 us (-26.2%)
ping_pong_oneshot smarm 1-thread 1619 -> 1367 us (-15.6%)
chained_spawn smarm 1-thread 481 -> 399 us (-17.0%)
Run-to-run spread on yield_many is ~3-4% (29565-30634), so the 22% move is
well outside the noise. Gap to tokio current_thread closes from ~2.4x to ~1.95x.
All tests green: cargo test --test context (4) plus the full suite (93 total).
The diagram claimed the entry/ret target sits at aligned_top-8 and r15 at
aligned_top-56. The code does (top & ~15) - 8 then a -= 8 before the first
write, so entry actually lands at aligned_top-16 and r15 at aligned_top-64.
Confirmed by single-stepping the shim's ret under llmdbg: the entry slot is
at an address with %16==0, giving the required rsp%16==8 on entry. Code was
correct; only the comment was wrong (and would mislead a maintainer).
A minimal Config::exact(1) program with two actors doing a fixed number of
yield_now() calls. No I/O/timers/channels, so the instruction stream stays
centered on the naked-asm switch. Frame-pointer-friendly. Intended for
single-threaded inspection under a debugger (e.g. llmdbg).
While running benchmarks, a hang surfaced in timer-only workloads — actors
sleeping with no IO in flight. Tracing it down, the race lives in sleep():
between the call to `timers.insert_sleep` and the subsequent `park_current`
yield, the actor is still in State::Runnable. If the timer fires in that
window, the old code's `if matches!(slot.state, State::Parked)` guard silently
drops the wakeup. The actor then parks normally and never gets re-queued —
it sleeps forever.
The fix mirrors what scheduler::unpark() and the IO FdReady path already do:
when the timer fires and the slot is still Runnable, set `pending_unpark`
instead of re-queuing immediately. The upcoming Park yield sees the flag and
re-queues the actor rather than suspending it, closing the race without any
new synchronisation.
Adds a regression test: 100 actors doing pure timer sleeps across ≥2
scheduler threads. The test asserts both correctness (all actors complete)
and timeliness (wall time < 2×sleep duration), which is enough signal to
catch a stuck actor even on a single-core CI runner.
Claude