Reframe the module doc in the gen_server house style: lead with what a
process group is and when to reach for one, the auto-eviction-on-death
behaviour, groups-vs-registry, and a running-context note. Keep the
runnable example; add an ignored dispatch/worker-pool example.
Move implementation reasoning to where a maintainer stands: lock
discipline onto the ProcessGroups store, the eager-cleanup rationale onto
reap_group, the join finalize-race detail into join's body comment.
Drop all RFC references and the stray phase marker, and lead the public
read/select fn docs with what the caller gets. Demote the pub(crate)
assert_type intra-doc link in pick_as to plain code, clearing a
pre-existing broken-link warning.
Apply the same explicit match+panic shape to the library layer (channel,
gen_server, gen_statem). Extend it to the smarm-trace-gated code that the
default `cargo clippy --lib` does not see: the GLOBAL lock-poison sites in
trace.rs and the current_pid sites inside te!() in channel.rs. Keep the
current_pid match inside the te!() argument so non-trace builds evaluate
nothing extra on the recv-wake hot path. const-init the trace thread-local.
Replace implicit unwrap()/expect() in the lock-ordered core with explicit
match arms. Lock-poison sites use one uniform message
("smarm: <lock> lock poisoned (core corrupt): {e}"); invariant sites panic
with a descriptive message naming the violated invariant. No behaviour
change: each rewrite preserves the prior panic-on-bad-arm semantics. Also
clears the accompanying clippy hygiene in these files (redundant_closure,
len_without_is_empty, too_many_arguments, unnecessary_sort_by,
missing_safety_doc, nonminimal_bool/unnecessary_unwrap).
A `=> postpone` row (cast/call/info) defers the current event untouched,
to be replayed after the next real transition. `handle` is now two-phase:
a borrow-only postpone pre-pass that hands the event back as
`Step::Postponed(ev)`, then the existing consuming `match (state, event)`.
The loop owns a FIFO queue, drained in the new state ahead of further
intake; a replayed event may postpone again. A postponed `call` keeps its
Reply, so a later state answers it.
`handle` returns `Step` (Postponed / Transitioned / Stayed) so the loop
can see both deferral and transition without reading the state cell.
`Resolution::Postpone` is removed: postpone is a pre-dispatch routing
decision, not a consuming-dispatch outcome.
Add the two timeout flavours, both surfacing as ordinary events matched
in on-state arms:
- cx.state_timeout(d): fires a state_timeout event after d in the current
state, auto-reset by the loop on every real transition.
- cx.timeout(name, d): fires a timeout(name) event after d, surviving state
changes, keyed by name, with cx.cancel_timeout(name).
Both ride the existing timer min-heap via send_after_to onto a new per-loop
system channel, selected above the inbox so a fire can't be starved by inbox
traffic. A local-id stamp on each fire lets a reset/cancel that loses the race
discard a stale fire. The macro grows an info: clause and folds three internal
Ev variants (Info, StateTimeout, Timeout) alongside cast/call, with new row
keywords info / state_timeout / timeout. Unmatched info silently drops (the
gen_server default); state/named timeouts have no default, so a state that can
see one must handle it or the match is non-exhaustive.
Rename the hand-written expansion-target example fused -> expanded, retire the
deprecated Switch demo machine (its round-trip / enter / panic-down coverage
moves onto the timer machine), and refresh the macro docs to the door machine.
cargo build --all-targets warning-free; cargo test green.
- rename StatemRef/StatemCallError/StatemSendError -> GenStatem*
- move the inline unit test out of src; consolidate the Switch coverage
onto a single macro-driven harness in tests/gen_statem.rs
- drop the redundant hand-written Switch test machine and the two
untracked rejected-direction probes (succ_enums, typed_edges)
- rename examples statem_{fused,macro}.rs -> gen_statem_{fused,macro}.rs
- strip RFC/chunk/spike provenance and fix the mislabeled "throwaway"
example header and dead cross-references
Declarative macro_rules! that fuses the hand-written statem surface into
one invocation: emits the unified event enum, the machine struct + state
cell, start, the Machine impl (dispatch + stay/transition apply-tail), and
the enter dispatch. User keeps the meaningful types, the per-state
successor enums, and the free handler fns.
Pure sugar: every safety property is a property of the emitted code,
checked by rustc, so a declarative macro carries (almost) the proc-macro
guarantee set:
1. forgotten (state,event) pair -> E0004 (total match, no injected _)
2. conflicting row -> unreachable_patterns (macro self-denies; HARD only
in-crate, suppressed cross-crate by in_external_macro -- documented)
3. orphan handler -> dead_code (handlers are user free fns)
4. out-of-set target -> E0599 (per-state successor enums)
Hygiene: bodies can't see the macro's self/cx, so the caller names them
via `context(data, prev, cx)` (shared call-site hygiene).
No separate transitions{} block: the match IS the table, successor enums
ARE the per-state target sets (fused variant, diverges from RFC
edge-lint).
- examples/statem_macro.rs: Door machine via the macro (parallel to the
hand-written examples/statem_fused.rs; diff the two to see the delta).
- in-crate test exercises a machine + anchors the in-crate #2 guarantee.
Runtime support layer for gen_statem, built against existing public API
(channel + scheduler::spawn), sibling to gen_server. No macro: per review,
build the primitives first and hand-write the Switch example to evaluate
whether a statem! macro earns its place before committing to one.
- src/statem.rs: Machine trait (on_start/handle), Resolution<S> with
From<S>, Cx (on_unhandled), Reply<T> move-only reply handle, StatemRef
(send/call), spawn + inbox loop. Real time only; Postpone + Cx timeout
arming are in the type surface but not yet acted on (chunks 2-3).
- examples/statem_switch.rs: the RFC Switch machine hand-written against the
primitives, tagged USER vs MACRO to mark what a macro would generate.
Asserts the RFC end-state (flips=1, enters=3).
- tests/statem.rs: call/cast round-trip, enter-on-start/transition-not-stay,
panicking-handler -> Down.
Reply<T> included (the call helper needs a handle type; keeps the example
true to the RFC surface) but isolated and trivially removable if we drop it.
These three RFC-015 tests call run() (the real runtime), so under
--cfg loom they construct loom atomics outside a loom::model block and
panic with "cannot access Loom execution state from outside a Loom
model". They are unit tests of runtime behavior, not state-machine
models. Gate the module #[cfg(all(test, not(loom)))], matching the
existing run_queue::tests precedent: they still run under normal
cargo test, and the loom build is now 28/28 clean.
A thin GenServer consumer of the Chunk-1 read primitive — the live
observer process (D12). ObserverRequest/ObserverReply are the wire
contract (D11); the version rides along on the snapshot/tree payloads,
which already carry SNAPSHOT_FORMAT_VERSION (D1). Behind the new
`observer` Cargo feature, off by default (D10): the primitive stays
always-on, only the transport is gated. Cast is Infallible, so the
server takes no async traffic and handle_cast is statically unreachable.
Gated integration test proves each verb relays exactly what the
corresponding primitive returns (snapshot/tree/actor_info), incl. a
forged-pid None and a live Parked classification.
Accumulate on-CPU cycles per actor as ActorInfo.budget_cycles, behind
the off-by-default budget-accounting feature (D6) — a reductions-style
work metric for relative comparison across runs.
- Approximate by design (per Mark): charge now - slice-start once at the
yield point, reusing the timestamp reset_timeslice already sets, so
one RDTSC per resume not two. Wake-slot resumes inherit the slice and
so slightly over-attribute the chain's time to the woken actor — noise
that averages out; we trade exactness for half the hot-path cost.
- Field/ActorInfo member are unconditional (keeps the snapshot shape
stable across the feature flag, D1); only the accumulation is gated,
so default builds are byte-identical and pay nothing. Reads return 0
when off. Single-writer Relaxed like the other counters; reset in
reset_counters (D7).
Matrix: default + feature-on + rq-mpmc + rq-striped + release + trace all
green; loom unaffected (feature off under --cfg loom).
Tally each dequeued message against the receiving actor, surfaced as
ActorInfo.messages_received (the 'is this actor draining slower than its
mailbox fills' signal).
- messages-received, not sent (D4): the receiver counts on its own
thread, so it's a single-writer Relaxed load+store on a hot Slot
AtomicU64, no atomic RMW (D5); reuses the stashed *const Slot from 2a.
- Incremented at all six channel dequeue-success sites (recv,
recv_timeout x2, recv_match, try_recv_match, try_recv) via
preempt::note_message_received; no-op outside an actor (null slot).
- Resets with overruns in reset_counters across the three lifecycle
sites (D7).
Matrix: debug default + rq-mpmc + rq-striped + release green; loom
slot_state/run_queue models pass (the 3 send_after_to loom failures are
pre-existing at fc014c4 — plain #[test]s under --cfg loom, not Chunk 2).
Tally overruns at the slice-expiry site in preempt.rs (the RFC 006
signal), surfaced as ActorInfo.overruns.
- Counter is a hot-region AtomicU64 on Slot, single-writer Relaxed
load+store (no atomic RMW), read Relaxed by the snapshot (D5).
- Reached from the rare expiry branch via a stashed *const Slot in a
preempt thread-local, set/cleared on the same resume/return boundary
as CURRENT_STOP — one TLS load, no runtime lookup. Shared infra for
the messages-received counter next.
- Reset in all three slot-lifecycle sites: Slot::vacant, reclaim_slot,
install_actor (standing invariant, D7) — per-incarnation counts.
tree() / tree_from() fold a Chunk-1 snapshot into a forest by grouping
each actor under its parent pid — one O(n) pass, no new reads. tree_from
is public so a held (or synthetic) snapshot can be folded without a
second scan.
- D8: actors parented at ROOT_PID are genuine roots; an actor whose
recorded parent is absent from the snapshot is re-rooted under the
sentinel and flagged orphaned, so the forest stays total. take()-on-
place doubles as a guard against re-entering a node.
- D9: the edge is parent/spawned-by, documented as not necessarily
supervision.
snapshot() / actor_info() return owned ActorInfo over the slab: pid,
names, fine-grained scheduling state, parent edge, trap flag, mailbox
depth, and monitor/link/joiner counts. Pure reads, no hot-path change.
- ActorState maps the packed slot word (no new storage); introspect.rs.
- D1: RuntimeSnapshot carries SNAPSHOT_FORMAT_VERSION from day one.
- D2: per-slot tearing (ps semantics); actor_info coherent per actor.
- D3: mailbox depth included. Registry channels now stored behind an
ErasedSender trait (downcast for clone_sender + type-erased
queued_len); depth summed over published channels under the registry
Leaf (Leaf -> Channel), kept out of the cold-lock pass so no two
Leaves are ever held at once. Depth covers published channels only.
- Done slots surface as root-less tombstones.
Add a cancellable message-delivery timer on the existing timer.rs min-heap,
the substrate the gen_server time idioms (idle/receive timeout, periodic tick,
debounce/backoff) need.
- Reason::Send { fire }: a type-erased delivery thunk. send_after (Pid<A>, via
send_to) and send_after_named (Name<M>, via send) capture dest+msg and
resolve the address on fire, not at arm time, so a dead target / restarted
name is observed when it fires; a failed resolve or closed inbox is dropped
(Erlang erlang:send_after semantics).
- Cancellation via an "armed" set keyed on the entry seq, exposed as an opaque
TimerId. Only Send timers use it; Sleep/WaitTimeout keep their inert-stale
behaviour untouched. pop_due fires a Send only while still armed and removes
it, so cancel returns true iff it landed before the fire (the race signal).
cancel is unscoped: any holder of the id can cancel (e.g. racing two servers
and cancelling the slow path).
- peek_deadline contract documented as "<= true next deadline" so a future
hierarchical timing wheel can back Timers without touching send_after or the
scheduler idle path. call_timeout left on recv_timeout (blast radius).
Tests: Timers-level (fire/cancel/race/clear/ordering) plus scheduler-level
delivery for both Name<M> and Pid<A>, cancel-prevents-delivery, and silent
drop on unresolved name / dead pid. Green on rq-mutex/rq-mpmc/rq-striped.
Implements the new addressing surface the examples in examples/ specify:
- spawn_addr::<A>(FnOnce(Receiver<A::Msg>)) -> Pid<A>: the typed-path
producer. Makes the inbox, spawns the body with its receiver, and publishes
the sender from the PARENT side (registry::install_for) before returning the
pid, so an immediate send_to always resolves (no race on the body installing
itself). Detached, like ServerBuilder::start.
- lookup_as / pick_as / members_as: re-type an erased pid as Pid<A> over the
heterogeneous registry/pg stores, sharing one helper (pid::assert_type).
Unchecked but sound: routing is by message TypeId, so a wrong A degrades to
SendError::NoChannel on the next send, never a misdelivery.
- dispatch::<A>(group, msg) -> Result<Pid<A>, SendError<A::Msg>>: pick_as +
send_to, with the message handed back on failure. New SendError::NoMember
variant for the empty/all-dead group case.
- gen_server naming: ServerName<G> backed internally by the registry's existing
typed-channel store keyed by TypeId::of::<Envelope<G>>() — Envelope stays
private, no separate directory. Type-state NamedServerBuilder<G> keeps the
current infallible ServerBuilder::start() untouched; its start() is fallible
(parent-side register, NameTaken). Free call/cast/whereis_server resolve per
use. ServerRef::shutdown (+ free shutdown) is the sys-style synchronous stop.
- root-exit teardown: the run's initial actor is recorded as root; when it
finalizes it flags root_exited, and the scheduler's idle verdict then stops
the parked-forever remainder (a one-shot RootDrain sweep). Deferred to the
idle point on purpose: the run queue drains first, so actors with queued work
finish rather than unwinding on the stop. This lets a named-server daemon (or
any pinned actor) wind the run down instead of hanging on live_actors > 0.
request_stop is refactored to a request_stop_inner core so the sweep can drive
it from inside the runtime without re-borrowing the thread-local.
Lib + tests + examples build warning-free; full suite green.
The explicit fallible fallback for when only an untyped Pid is in hand (off a
Down, out of a future members()), so the typed send_to / send-by-name stay the
default and callers don't grow workarounds around a missing primitive.
- send_dyn::<M>(Pid, msg): one line over the shared send_to_pid core. Same
identity-bound, no-redirect liveness as send_to (Dead once the incarnation is
gone); the difference is the explicit M, so NoChannel is a real runtime
outcome here rather than the debug_assert it collapses to on the typed paths.
- Takes Pid<Erased> specifically: a typed Pid<A> must .erase() to reach it, so
opting into the fallible downcast stays visible at the call site.
tests/registry.rs: send_dyn_delivers_and_reports_wrong_type (right type
delivers; live actor with no channel for the asked type -> NoChannel) and
send_dyn_to_dead_pid_is_dead. Full suite + order-checker green, warning-free.
This completes the RFC 014 send surface: Name<M> (re-resolving), Pid<A>
(typed direct), send_dyn (bare-pid). send_after (§6) can now target any of
them — its Dest question is answered.
Make a Pid<A> messageable: install + send_to, the direct addressing mode that
dies with the actor and never redirects (the counterpart to the re-resolving
Name).
- install::<A: Addressable>(tx: Sender<A::Msg>) -> Pid<A>: opt-in, lazy,
nameless publish (RFC 014 §5). Files the actor's inbox into the by_index
mailbox table under TypeId::of::<A::Msg>() and re-types self's identity as
Pid<A>. Infallible: no name to collide on, self is always live in run().
- register and install now share publish_channel (the insert-or-extend-mailbox
step factored out); register additionally binds the name. Byte-identical
storage, so name- and pid-addressing populate the same table.
- send_to::<A>(Pid<A>, A::Msg): resolve by raw pid, deliver with NO redirect.
The stored mailbox must be this exact incarnation (generation included) and
live, else SendError::Dead — even when the slot now holds a different live
actor, which is left untouched. Resolution clones the Sender under the Leaf
lock, releases, then sends (Leaf -> Channel), as name send / pg / finalize.
- send_to_pid is the shared raw-pid core; send_dyn (§4.6) lands on it next.
- SendError gains Dead(M), the pid-path counterpart to name-path Unresolved;
into_inner / Display / Debug updated. Name send never yields Dead, pid send
never yields Unresolved — disjoint by construction, documented on the enum.
tests/registry.rs: install_then_send_to_pid_delivers; and the load-bearing
send_to_does_not_redirect_after_takeover (slot reused by a new incarnation, the
stale Pid<A> send returns Dead and the new occupant gets only its own message).
Full suite + order-checker green, warning-free across all targets.
Fold the made-up Addr<A> back into the pid, where it belonged. The typed
handle IS the pid now.
- RawPid { index, generation }: today's Pid, renamed — the raw numbers, the key
for identity-only plumbing and the heterogeneous monitor/link/pg tables.
- Pid<A = Erased>: RawPid + phantom actor type. Both an identity and a direct,
identity-bound address; when A: Addressable a send delivers A::Msg to exactly
this incarnation (next phase). Hand-written impls (eq/hash/fmt on the raw
only, no A bounds); fn()->A phantom keeps it Copy+Send+Sync for any A.
- Erased: uninhabited, deliberately NOT Addressable, so a typed send to a
Pid<Erased> won't compile — that's what send_dyn (section 4.6) will be for. It
is the default type arg, so every plain "Pid" written today still compiles as
Pid<Erased>; that, plus the fact that nothing destructures pid fields, is why
an identity change touching 18 files cascaded to zero internal edits.
- Addr<A> deleted; Name<M> / the Phase-2 registry unchanged (they key on raw).
Per the call to make the consumers take typed pids: the user-facing identity
APIs — monitor, link, unlink, request_stop, spawn_under(supervisor), pg::join,
pg::leave — are now generic over A and erase at the boundary; their bodies are
byte-for-byte unchanged. Pure plumbing (unpark, set_current_pid,
register_supervisor_channel) stays erased — it only ever sees internal identity.
Phase 1's Addr tests become Pid<A> tests (identity/copy/erase/Debug, Send+Sync).
Full suite + order-checker green, warning-free across all targets.
Rip out the old name<->pid bimap; rebuild registry.rs as the live mailbox
directory. A name resolves to a SINGLE actor (many-per-label is pg's job); that
actor owns a SET of typed channels (channels are typed, so no single mailbox),
keyed by message TypeId. Resolution: name -> pid (one actor) -> Mailbox ->
channel for type M. Same name + different type parameter selects different
channels of the one actor, so capability separation (§4.7) needs no new type.
- register<M>(Name<M>, Sender<M>): captures the current actor's channel under a
name (one step, per decision). Adds channels cumulatively; NameTaken only vs a
different LIVE holder; stale/dangling bindings pruned on contact.
- send<M>(Name<M>, msg): the payoff — resolve, clone the Sender UNDER the Leaf
lock, release, then send (Leaf -> Channel; a send can unpark). Returns the msg
on Unresolved / NoChannel / Closed.
- whereis -> single live pid; unregister frees only the name (mailbox/other
names survive). name_of (reverse lookup) dropped — deferred to introspection.
- Contained erasure: each channel is Box<dyn Any+Send> filed under TypeId::of M
and downcast to its own keying type, so the downcast can't fail on good data
(debug-asserted via the stored type_name). Phantom M on Name re-imposes type.
- One Leaf RawMutex (the fold), so a name send stays on a single Leaf — two
Leaves at once is a hard panic. runtime.rs field/init unchanged (same Registry
name + new()).
tests/registry.rs rewritten for the new semantics: send-by-name delivery,
many typed channels on one actor, same-name/different-type routing, NameTaken,
takeover across slot reuse, Unresolved/NoChannel. Also fold in a Phase 1 fixup:
the phantom test key was an enum whose variants tripped dead-code under the test
build (only caught now that I run -D warnings on --tests). Full suite +
order-checker green, warning-free across all targets.
Add the value-token layer for typed addressing, ahead of the handle table:
- Addressable { type Msg }: minimal actor-type -> message-type hook (decision
(a)). Raw actors are closures over channels; GenServer is multi-message and
stays addressed via ServerRef. This gives the single-message actors a key.
- Addr<A>: direct, identity-bound address (RFC's Pid<A>, renamed to avoid the
collision with the non-generic Pid identity). Plain Pid + PhantomData<fn()->A>.
- Name<A>: durable, re-resolving address; const-constructible (Name::new).
Both tokens are Copy + Send + Sync regardless of key (phantom is fn()->T), with
hand-written trait impls so no spurious T: Copy/Eq bounds leak in. No behavior
yet; send/resolve/storage land in later phases. Addr::new is #[cfg(test)] until
Phase 2 has a real (non-test) caller. Unit tests cover identity/copy/eq/debug
and Send+Sync.
A plain-actor example showing the one thing process groups are for: a live
membership view that self-heals on death. Two workers join a group; one dies
and is evicted by the death hook with no deregistration call; one leaves
voluntarily. Runs under cargo test --doc.
(Deliberately not a worker-pool-with-dispatch example: pg is membership and
liveness, not a message router — routing a job to a picked pid needs a
pid->handle map that pg does not provide, so a dispatch example would document
the adapter, not pg.)
members/pick now filter through a lock-free, generation-checked slot-word
read (live(), identical to the registry's guard) so a member that is already
dead but whose Down has not yet been drained is never *returned*. Eviction
stays the monitor's job (reap_group); this is the belt-and-braces read guard.
- members_where / first_member_where take an is_live oracle and replace the
unconditional first_member; members_of becomes test-only (raw enumeration).
- Unit test: the backstop hides a member the monitor has not yet reaped, and
does not itself evict (storage still holds it until reap).
Full suite green, warning-free.
Wire eviction to actor death via the monitor subsystem, using drain-on-
contact (no scheduler hot-path hook).
- Each membership now carries its own Monitor alongside the group entry;
join() installs monitor(pid) (registration races finalize_actor under the
cold lock, as every monitor does) and a redundant join tears its extra
monitor back down.
- reap_group: every group op drains the touched group's monitors with a
non-blocking try_recv (a delivered Down or closed channel = dead) and, on
the first death, sweeps that pid out of *every* group via remove_where —
so a death vanishes from all its groups on next contact.
- Lock discipline: monitor()/demonitor() (cold = Leaf) run outside the group
lock; try_recv (Channel) runs under it, permitted by the Leaf->Channel
order; evicted/rejected monitors are dropped after the lock releases so no
receiver-drop wakeup runs under it. Validated by the debug-build lock-order
checker passing under the integration tests.
- remove_where now returns the evicted monitors (for deferred drop) and
preserves intra-group insertion order.
- Tests: unit (synthetic monitors) for reap live/dead/closed + sweep;
integration (real actors) for death-vanishes-from-every-group, pick on an
all-dead group, peer survival, leave isolation, dead-at-join eviction.
Outstanding: a miri pass (cargo +nightly miri test --lib pg::) is untried —
first build exceeds the sandbox window and the futex RawMutex may need a miri
shim; the mechanical Leaf->Channel checker is the primary soundness guard.
Add a process-groups module (name -> multiset<Member>) sibling to the
registry, plus the cluster-shaped identity it is keyed on.
- NodeId/Incarnation u32 newtypes; Member { node, incarnation, pid } at
final (BEAM NEW_PID_EXT-shaped) layout.
- Config::node_id/incarnation builder setters beside wake_slot, defaulting
to a fixed single-node value; threaded through RuntimeInner::new.
- process_groups: RawMutex<ProcessGroups> on RuntimeInner, Leaf class,
mirroring the registry field.
- remove_where: the one dumb predicate-eviction primitive (no liveness wired
yet; first caller is the Phase 2 death hook).
- Public surface (join/leave/members/pick) in pre-liveness/pre-monitor form,
Pid-shaped; node/incarnation filled from runtime identity.
- Structural unit tests on synthetic members: idempotent join, multiset
semantics across groups, generation-distinct members, leave + empty-group
pruning, predicate sweep, incarnation-sweep shape.
request_stop sets the flag and issues a wildcard unpark, but unpark
no-ops on a Queued actor (the pending run "is" the wake). If the
actor's first action on resume is a blocking park — no allocation, no
check!() on the way — the wake-side-only check in park_current is
unreachable: the actor parks with the stop flag already set and no
wake is ever coming. The runtime then idles forever.
Fix: check_cancelled at the ENTRY of park_current, before the switch.
The remaining window (flag lands after the entry check, before the
park) is covered by the existing protocol: the stop's unpark finds
Running / the prep-to-park window, sets Notified, and the park-return
re-queues into the wake-side check. Unwinding from the entry is the
same unwind path as the wake side; leftover wait registrations are
stale-epoch / dead-generation and self-clean at their wakers' failed
CAS.
This is bug #1 of the urus chunk-2 session (bug #2, the terminal
wake, is the preceding commit). Repro: tests/cancel.rs
stop_flagged_while_queued_lands_at_first_park — deadlocks without the
fix (watchdog-bounded), passes in <50ms with it.
An idle scheduler thread blocks in poll_wake on a snapshot of
peek_deadline / io_outstanding. Only the io threads ever write the wake
pipe (on completion push); enqueue does not. Mid-flight that is masked —
the thread that caused an enqueue is awake and processes it — but at
termination the snapshot can go terminally stale, two ways:
- Stale io_outstanding (infinite hang): an actor parked in wait_readable
is request_stop'ped. Cancellation deregisters the waiter and produces
NO completion, so the wake pipe is never written. A sibling blocked on
io_outstanding > 0 with no timers pending sits in poll(-1) forever.
- Orphaned timer deadline (finite stall): an actor cancelled out of a
long sleep leaves its timer entry behind. Clearing the wheel at
AllDone doesn't help a sibling that already blocked on that deadline;
it sleeps it out in full.
In both cases the remaining actors finish on the other scheduler thread,
which reaches the live==0 && io_out==0 verdict and returns without
waking the blocked one. Runtime::run then stalls or hangs in its worker
join. Found via urus's graceful-shutdown tests (~10% flake, one test per
variant); confirmed by gdb dumps of hung processes showing run() in
JoinHandle::join over a sibling in poll(wake_fd, 59750ms) resp. poll(-1).
Fix: Io::wake() writes the wake pipe directly; the AllDone arm calls it
after the timer clear. One byte wakes every poller; each re-runs the
verdict, independently reaches AllDone, and re-wakes — idempotent.
tests/terminal_wake.rs reproduces both variants deterministically: the
root busy-spins (no timer entries, occupies one scheduler thread) so the
sibling settles into the stale idle wait before the stop is issued. Both
hang without the fix and pass in <0.5s with it.
Known residual gap, deliberately unfixed: the timers-pending /
no-io-subsystem idle branch blocks in thread::sleep and has no wake
mechanism at all — the same stall exists for runtimes that never
initialize io. Roadmap candidate alongside cross-thread unpark and
entry-side check_cancelled in park_current (the lossy-QUEUED-stop bug).
Adding slot_hits + slot_displacements in RFC 005 grew SchedulerStats
from 16 to 32 bytes — exactly 2 per cache line. run_queue_len (written
on every push/pop by thread N) then false-shared with fields of thread
N+1, causing ~45-60% yield-storm regression at t≥8 visible equally in
slot-on and slot-off paths.
A thread-local Cell<Option<Pid>> per scheduler, checked before the shared
run queue. Runtime-selected via Config { wake_slot: bool }, default OFF
until the slot shootout accepts it (one binary benches both arms).
Push policy: slot-eligible iff the wake originates from actor context
(current_pid().is_some()) — the slot push replaces run_queue.push at the
tail of the unpark protocol's Parked → Queued CAS, so at-most-once-enqueued
holds verbatim as (slot ⊕ shared queue). Scheduler-context wakes (timer/IO
drain) and spawns always go shared (spawns never reach unpark_inner at all).
Displacement is Go semantics: newest wake takes the slot, occupant pushed
shared — moved, never copied.
Pop order: slot, then shared. Slot-popped actors skip reset_timeslice() and
inherit the waker's remaining slice; a handoff chain is bounded by one
slice, after which the preempt-yield re-enqueue goes shared (a yield is not
a wake) — the one-slice starvation bound, zero new counters. Idle and
AllDone are only reachable with an empty local slot by pop order; an
occupied slot elsewhere holds a Queued (live) actor, so the counter-first
termination argument is untouched.
Observability: per-thread slot_hits / slot_displacements (reset at run()
start so post-run stats() reads are per-run), SlotPush/SlotPop trace events.
Bench plan (roadmap v0.9 item 2): rq_runtime gains the slot on/off
dimension (SMARM_BENCH_SLOT, default "0 1") — ping-pong-pairs is the
target metric, yield-storm the regression guard, spawn-storm the
neutrality check. RQCSV grows a slot column; RQSLOT lines carry the
counters; bench_rq.sh aggregates both. Tests pin the push policy through
the counters (actor-context hits, spawn/join bypass, displacement,
default-off, per-run reset).
FdArm composes fd readiness with channel arms on one wait epoch.
Selectable grows fallible sel_register and an eager-cleanup hook;
losing/stop-unwound/timed-out fd arms are unregistered (waiters entry
+ kernel ONESHOT) so the fd is never poisoned. try_select /
try_select_timeout surface registration errors (EBADF, EMFILE,
AlreadyExists) instead of RFC 008's permanently-ready lean, which
would busy-loop a healthy-but-unregistrable fd; select/select_timeout
stay infallible for channel-only arms. Adds wait_readable_timeout /
wait_writable_timeout as one-arm selects.
Known benign race (pre-existing, slightly widened): a queued FdReady
racing the cleanup DEL can spuriously wake a fresh waiter on that fd;
absorbed by select's defensive re-loop. Fixable by epoch-stamping
completions.
A stopped actor unwinding out of wait_fd's park leaked its waiters entry
and the kernel-side EPOLLONESHOT registration; the stale entry then failed
every future wait_*() on that fd with AlreadyExists (the defensive bare DEL
in epoll_register sits behind the contains_key check, so it never ran).
Fix where the invariant breaks: a drop guard in wait_fd, armed after a
successful register and forgotten on the normal wake (where FdReady already
removed + DEL'd). On unwind it cleans up iff the entry is still this wait's
(pid, epoch) — an entry consumed by a racing FdReady means the fd may carry
another actor's fresh registration, which must be left alone. Closes the
v0.2 fd-hygiene TODO.
Monitors are created at runtime (you watch a worker you just spawned in a
handler), so down arms can't ride the static info list. init grows a
&ServerCtx parameter (breaking; no-op default) whose clonable Watcher hands
Monitors to the loop over a control arm; the loop selects the live down
arms ahead of everything else. Arm priority: downs → control → infos →
inbox. A delivered Down retires its arm (monitors are one-shot); a state
that never clones the Watcher closes the control arm after init and the
loop falls back to the plain-inbox park.
type Info + handle_info (no-op default) on GenServer; ServerBuilder
(with_info/under/start) so start variants don't multiply — start/start_under
stay as wrappers. The loop selects [infos.., inbox] in priority order when
info arms exist, and keeps the plain recv() park when none do. Closed info
arms are silently removed (closed-arm-is-ready-forever); a closed inbox
still means graceful shutdown.
The deadline is one more stamped waker on the select's wait epoch: a unit
TimerTarget whose on_timeout is a bare unpark_at. Nothing registers in any
arm for it, so there is nothing to cancel or leak — an arm winning leaves
the timer entry to die at its epoch CAS; the timer winning leaves the arms'
registrations to self-clean like any select loser's. Classification is pure
channel state (wakes are precise): some arm ready -> Some(first, priority
order); none -> the timer was the only remaining stamped waker -> None.
Message-first on a raced deadline, same as recv_timeout.
Registration pass factored into register_arms, which owns the retire_wait
obligation on the ready-now exit for both entry points.
select(&[&dyn Selectable]) -> usize registers (pid, epoch) in every arm
under one wait epoch, parks once, and returns the first ready index in
documented priority order (BEAM-style; no fairness promise). A closed arm
counts as ready — and stays ready forever, so callers drop it from the set
once observed. Losing arms need no cancellation pass: the winning wake
consumed the epoch, so their registrations die at their wakers' failed CAS
or are overwritten by the receiver's next wait on that channel (the
single-receiver debug_asserts relax to 'none or own pid' accordingly).
The one genuinely new protocol piece is the no-park exit: returning with an
arm ready at registration time leaves earlier arms holding LIVE-epoch
registrations, whose wakes could fault a later one-shot park as a pending
notification. scheduler::retire_wait closes it — bump the epoch (in-flight
wakes die at their CAS), eat a notification that already landed
(StateWord::clear_notify), then re-observe the stop flag, in that order.
Proved by the new loom theorem retire_eats_late_arm_notification; the
integration probes in tests/select.rs fire stale loser-arm wakes and assert
a subsequent sleep's one-shot park holds its full duration.
The slot epoch is THE wait identity, so the hand-rolled per-primitive copies
go away: channel loses cur_wait/next_wait_seq/timed_out, mutex loses Wait.seq
and next_seq, TimerTarget::on_timeout takes the epoch. Registrations become
(pid, epoch) — channel parked_receiver, mutex waiters, io fd waiters,
Blocking io completions, sleep timers, joiner lists — and their wakers move
to unpark_at. begin_wait is lock-free, so each primitive opens the wait
inside the same critical section that publishes the registration.
recv_timeout's wake-classification loop collapses: wakes are precise, so
queue → Ok, senders==0 → Disconnected, else Timeout — the 'Defensive'
re-register branch is now unreachable by protocol, not by audit. Same for
Mutex::lock_timeout's one-shot park.
End-state invariant, auditable in one sentence: the only wildcard wake is
request_stop, which is terminal.
smarm-agent