From 5428697f9e8b5ffffb6ea16d7ba45b4754bd2e4a Mon Sep 17 00:00:00 2001 From: smarm Date: Tue, 9 Jun 2026 21:53:01 +0000 Subject: [PATCH] docs(roadmap): consolidate task.md + ROADMAP_v0.5.md into ROADMAP.md Single source of truth: v0.4 history, v0.5 phases, v0.6 open tracks, and invariants/gotchas. Old task.md and ROADMAP_v0.5.md removed. --- ROADMAP.md | 186 ++++++++++++++++++++++++++++++++++ ROADMAP_v0.5.md | 157 ----------------------------- task.md | 263 ------------------------------------------------ 3 files changed, 186 insertions(+), 420 deletions(-) create mode 100644 ROADMAP.md delete mode 100644 ROADMAP_v0.5.md delete mode 100644 task.md diff --git a/ROADMAP.md b/ROADMAP.md new file mode 100644 index 0000000..f4d0fba --- /dev/null +++ b/ROADMAP.md @@ -0,0 +1,186 @@ +# smarm — Roadmap + +## v0.4 — Actor primitives ✅ DONE + +### 1. Cooperative cancellation ✅ (`a8ddb4a`) +Stop flag on `Actor` behind `Arc`. Observation points: +`maybe_preempt`/`check!()` + the wakeup side of `park_current`/`yield_now`. +Sentinel = `StopSentinel` (zero-size), caught in trampoline → `Outcome::Stopped`. +`join()` on a stopped actor returns `Ok(())`. + +Known gap (by design): a tight no-alloc loop without `check!()` cannot be +stopped, same as preemption. + +### 2. one_for_all / rest_for_one + ordered shutdown ✅ (`351dc9c`) +`Strategy::{OneForOne,OneForAll,RestForOne}` via `.strategy()`. Triggering +child's `Restart` policy decides whether anything restarts; strategy decides +which siblings are cycled. Survivors stopped in reverse start order, awaited on +the existing `supervisor_channel` funnel, restarted in start order. + +`Signal::Stopped(pid)` + `DownReason::Stopped` added (distinct from Exit). + +Also fixed a latent bug (`e80334b`): orphaned timers were gating shutdown. +`live == 0` is now the shutdown condition; timer heap cleared on exit. + +### 3. Links + trap_exit ✅ (`6581484`) +`Slot.links: Vec` (bidirectional). On finalize: abnormal death → +`request_stop(peer)` unless peer traps, in which case deliver `ExitSignal` +message. Normal exit does not propagate. `trap_exit()` returns a dedicated +`Receiver` (distinct from monitor `Down` channel). + +### 4. Selective receive ✅ (`03f3875`) +`Receiver::recv_match(pred)` + `try_recv_match`. Scans `VecDeque` front-to-back, +removes+returns first match, parks and re-scans on new arrivals. `pred` is +`Fn(&T) -> bool` (not `FnMut`; runs under the channel lock — keep it cheap). + +### 5. Grab-bag ✅ +- **demonitor** (`5181037`): `monitor()` → `Monitor { id, target, rx }`. + `demonitor(&m) -> Option`. Dropped `Sender` released post-lock + (non-reentrant shared mutex discipline). +- **gen_server** (`55221e9`): `GenServer` trait, `call`/`cast` over one + `Envelope` inbox. `terminate` hook via drop guard. No `handle_info`, no call + timeout (deferred — needs cross-channel mailbox merge or per-recv deadline). +- **Named registry**: `register`/`whereis`/`send_by_name` — **not yet done**. + HashMap in `SharedState`. No dependencies; can land any time. +- **arm-port branch**: aarch64 context-switch backend extracted into + `src/arch/`. Untested on hardware. Merge only after on-device validation. + +--- + +## v0.5 — Runtime decomposition & run-path scaling ✅ DONE + +Goal: dismantle the single `Mutex` along the run path so the +runtime scales to dozens of cores, with crash isolation hardened so a torn +write or a stray cancellation can never poison shared runtime state. + +Guiding rules established this cycle: +- **Lock count that matters is hot-path locks.** Cold lifecycle paths may keep a + small per-slot lock; the run path targets zero locks. +- **Lock ordering: io-before-shared, slot locks are leaves** (never hold two + slot locks at once). +- **No unwind inside a runtime critical section.** Stop sentinel only fires at + lock-free observation points. +- **Never hold a thread-local guard across a possible switch point.** An actor + can be preempted at any allocation and resume on a different OS thread. + `with_runtime`, `with_shared`, and every `RawMutex` guard disable preemption + for this reason. + +### Phase 1 — State decomposition ✅ (`3c7e26b`) +`next_monitor_id` → `AtomicU64`. `timers` → own `Mutex`. `io` → own +`Mutex>`. `pending_closures` → folded into `Slot::pending_closure`. +`check_cancelled()` gated behind `PREEMPTION_ENABLED` (poison fix). + +`SharedState` now holds only: `slots`, `free_list`, `run_queue`, `root_pid`. + +### Phase 2 — Slot table split ✅ (`5e0c9d4`) +Fixed slab `Box<[Slot]>`. Generation packed into state word: one +`AtomicU64 = (gen << 32) | state` — gen check is atomic with every transition. +Per-slot CAS state machine `Vacant/Queued/Running/RunningNotified/Parked/Done`. +Per-slot `RawMutex` for cold collections. Free list → `RawMutex>`. +`live_actors` atomic; termination = io_out==0 && queue empty && live==0. + +### Phase 3 — Pluggable run queue ✅ (`1b3b618`) +Compile-time selected via feature flags. Variants: +- `rq-mutex` — `Mutex`, baseline. +- `rq-mpmc` — hand-rolled Vyukov bounded MPMC ring. +- `rq-striped` — M Vyukov rings, fetch-add ticket distribution. + +All variants compile in every build; feature only picks the alias. Bounded rings +are sound via slab cap + at-most-once-enqueued invariant. + +Queue ops require preemption disabled (a producer suspended mid-publish stalls +every consumer behind its cell — livelock). + +### Phase 4 — Bench harness ✅ (`6d9f369`) +`benches/rq_micro.rs`: raw structure sweep. `benches/rq_runtime.rs`: +yield-storm, ping-pong-pairs, spawn-storm. `scripts/bench_rq.sh`: rebuilds per +feature, aggregates to `bench_results/summary.csv`. + +### Phase 5 — Safety hardening & model checking ✅ (`039703d`) +State machine extracted to `src/slot_state.rs` for loom checking. Loom on the +rings via `src/sync_shim.rs`. `NoPreempt`/no-unwind/TLS-guard audit complete. +Invariant-assertion sweep: every `StateWord` transition self-asserts its +precondition. + +--- + +## v0.6 — Fast follows & open tracks + +Priority order within the cycle; each item is its own commit. + +### 1. Channel mutex migration (first change) +`channel::Inner` holds a `std::sync::Mutex`. Phase 2 surfaced a sharp +correctness issue: the `MutexGuard` is held with preemption **enabled**, so a +timeslice switch inside a channel critical section migrates the actor and +releases the pthread mutex from a different OS thread — UB (Linux futexes happen +to tolerate it, but it's not guaranteed). The Phase-1 `check_cancelled` gating +already removes the unwind source, so poison is safe today; this is about the +cross-thread release. + +Migrate to `RawMutex` (which disables preemption and is cross-thread-release +sound by construction). `recv_match` runs a user predicate under this lock — +keep it cheap/pure. + +### 2. Named registry +`register(name, pid)` / `whereis(name) -> Option` / `send_by_name`. +`HashMap` in `SharedState`, guarded by the shared lock. No +architecture changes; can land immediately after #1. + +### 3. gen_server: handle_info + call timeout +- `handle_info`: needs a cross-channel mailbox merge (call/cast inbox + + out-of-band signals). The still-unmade decision: whether to introduce a + unified per-actor mailbox or keep per-channel `recv_match` composition. +- Call timeout: `call` with a deadline — monitor the server, wait + reply-or-Down-or-deadline, then `demonitor`+drop so a timed-out call leaks + no registration. Requires a per-`recv` deadline / `Signal::Timeout` primitive + that doesn't exist yet. + + These two can land independently; call timeout is smaller and more useful + first. + +### 4. Unbounded / configurable-bounded actor count +v0.5 ships a fixed slab with a loud assert (`Config::max_actors(n)`, default +16 384). Revisit with a segmented slab (array of `AtomicPtr`, doubling +segment sizes, append-only) once the cap is actually hit or a workload demands +it. Do not let the fixed cap calcify into an assumption. + +### 5. Idle-wakeup eventcount +Idle scheduler threads currently poll-sleep at 100 µs. A futex-based eventcount +would reduce idle latency. Defer until benches show it matters. + +### 6. IO fd hygiene on actor death +Pre-existing v0.2 TODO in `io.rs`: fds registered with epoll for a dead actor +are not cleaned up. Audit and fix. + +### 7. arm-port validation & merge +`arm-port` branch carries an AAPCS64 context-switch backend, never run on +hardware. Steps: build + run full test suite on an aarch64 device; check +`chained_spawn` / `yield_many` bench medians for regressions vs x86 baseline; +merge and update README module table to reflect multi-arch support. + +--- + +## Invariants & gotchas (respect these across all cycles) + +- **Shared mutex is non-reentrant.** `Sender::send` can call `unpark` → + `with_shared`. Never send on a channel while holding the shared lock. Pattern: + `mem::take` data under the lock, send after releasing. See `finalize_actor`. +- **`finalize_actor` order:** take stack/waiters/monitors under lock + set + Done/outcome → recycle stack → deliver supervisor Signal + monitor Downs → + unpark joiners → reclaim slot if `outstanding_handles==0`. Death notifications + always precede reclamation. +- **Slot lifecycle reset in THREE places:** `Slot::vacant()`, `reclaim_slot()` + (runtime.rs), slot-init block in `spawn_under` (scheduler.rs). Any new `Slot` + field must be reset in all three. +- **Pid = (index, generation).** Stale handles caught by generation mismatch in + `slot()/slot_mut()`. The monitor `NoProc` path relies on this. +- **No `select`, no unified per-process mailbox.** Why the supervisor uses the + single `supervisor_channel` funnel. `recv_match` is per-channel only; a + cross-channel merge is the still-unmade decision (see v0.6 #3). +- **Cooperative-only.** Preemption and cancellation both depend on the actor + reaching `check!()`/yield/alloc/blocking points. +- **Queue ops require preemption disabled.** A producer suspended mid-publish + stalls every consumer — livelock. `with_runtime`, `with_shared`, and + `RawMutex` guards all disable preemption for their span. +- **`run()` is single-thread** (`Config::exact(1)`); tests rely on deterministic + single-thread ordering. Multi-thread via `runtime::init(Config…)`. diff --git a/ROADMAP_v0.5.md b/ROADMAP_v0.5.md deleted file mode 100644 index 2b62af5..0000000 --- a/ROADMAP_v0.5.md +++ /dev/null @@ -1,157 +0,0 @@ -# smarm v0.5 — Runtime decomposition & run-path scaling - -Goal: dismantle the single `Mutex` along the run path so the -runtime scales to dozens of cores, with crash isolation hardened so a torn -write or a stray cancellation can never poison shared runtime state. - -Guiding rules established this cycle: -- **Lock count that matters is hot-path locks.** Cold lifecycle paths - (spawn/join/monitor/link/finalize) may keep a small per-slot lock; the run - path (yield/park/unpark/pop/resume) targets zero locks. -- **Lock ordering: io-before-shared, and slot locks are leaves** (never hold - two slot locks at once). Any new lock states its position in this order. -- **No unwind inside a runtime critical section.** The stop sentinel only - fires at lock-free observation points. -- **Never hold a thread-local guard across a possible switch point.** An - actor can be preempted at any allocation and resume on a different OS - thread; a live `RefCell` borrow (or std `MutexGuard`) then pairs its - acquire/release across two threads' locals — count underflow / UB. - `with_runtime`, `with_shared`, and every `RawMutex` guard disable - preemption for exactly this reason (phase 2 found this the hard way). - ---- - -## Phase 1 — State decomposition: easy peel-offs ✅ DONE -Split independent concerns out of `SharedState` so the global lock guards less. -- [x] `next_monitor_id` → `AtomicU64` on `RuntimeInner` (lock-free id minting). -- [x] `timers` → own `Mutex` (only drain-winner + blocking prims touch it). -- [x] `io` → own `Mutex>` (completion queue already self-locked). -- [x] `pending_closures: Vec` → folded into `Slot::pending_closure` (per-actor data). -- [x] Termination check split: read io liveness *before* `shared`; ordering - argument documented in `schedule_loop`. -- [x] **Poison fix:** gate `check_cancelled()` in `maybe_preempt` behind - `PREEMPTION_ENABLED`, so a cancellation sentinel can never unwind while a - `std::sync::Mutex` (shared/channel) is held. Regression test: - `tests/poison_stop.rs`. - -`SharedState` now holds only: `slots`, `free_list`, `run_queue`, `root_pid`. - -## Phase 2 — Slot table split ✅ DONE -Make slot lookup lock-free and per-slot state independently mutable. -- [x] Fixed slab: `Box<[Slot]>`, slots never move → stable - addresses, lock-free index. **Assert on exhaustion** with a panic message - naming `Config::max_actors(n)` as the fix. Default `max_actors = 16_384`. - (Mental note: must become unbounded or configurable-bounded later — see - "Deferred" below. Do not let the fixed cap calcify into an assumption.) -- [x] Generation packed INTO the state word (better than the planned separate - `AtomicU32`): one `AtomicU64 = (gen << 32) | state`, so the gen check is - atomic with every transition — no ABA, no spurious unparks, by CAS. -- [x] Per-slot CAS state machine `Vacant/Queued/Running/RunningNotified/ - Parked/Done` replacing `state` + `pending_unpark`. Also fixed a latent - lost wakeup in the Blocking-IO completion path (result set for a - still-Running actor without a flag). -- [x] `sp` → relaxed `AtomicUsize`; stop flag + first-resume closure as - `AtomicPtr`s — resume path fully lock-free. -- [x] Per-slot raw non-poisoning futex mutex (`src/raw_mutex.rs`) for the - cold collections. Guard enters `NoPreempt`. -- [x] Free list → `RawMutex>` (a leaf lock). Treiber/striped only if - the phase-4 spawn-storm bench shows contention — revisit then. -- [x] `finalize_actor` link cascade locks peers one at a time; acyclicity - argument written at the site. `link()` registers target-first with the - race argument at the site. -- [x] `live_actors` atomic; termination = io_out == 0 (read pre-queue-lock) - && queue empty && live == 0; decrement-last ordering documented as the - correctness crux at the site. - -## Phase 3 — Pluggable run queue ✅ DONE (shootout = phase 4 harness) -- [x] `RunQueue` alias in `src/run_queue.rs`, compile-time selected, - `compile_error!` guards for zero / >1 features. No runtime dispatch. - All variants compile in every build (unit tests always run); the - feature only picks the alias. Non-default variants: - `--no-default-features --features rq-…`. - - `rq-mutex` — `Mutex`, the control/baseline. - - `rq-mpmc` — single hand-rolled Vyukov bounded MPMC ring - (per-cell seq numbers). Strict FIFO; "one hot cache line". - - `rq-striped` — M Vyukov rings, fetch-add ticket distribution. Relaxed - FIFO, reordering bounded by ~M. Predicted winner @20c. -- [x] Bounded rings sound via slab cap + at-most-once-enqueued (occupancy ≤ - `max_actors`); mpmc capacity = next_pow2(max_actors), striped - Σcapacity ≈ 2×max_actors with probe-from-home-stripe. A full ring - panics as an invariant violation (double enqueue), never spins. -- [x] All hand-rolled, dependency-free. -- [x] Two contracts surfaced by the extraction, documented + debug-asserted - in `run_queue.rs`: queue ops require preemption disabled (a producer - suspended mid-publish stalls every consumer behind its cell — livelock); - and pop-None is a snapshot, not a fence — termination is counter-first - (`live == 0` alone implies the queue holds nothing actionable; argument - rewritten at the `schedule_loop` site). - -## Phase 4 — Bench harness ✅ DONE (harness; real numbers from the 20-core box) -- [x] `benches/rq_micro.rs`: raw structures, threads × p:c ratio sweep. One - binary covers all three structures (types compile in every build). -- [x] `benches/rq_runtime.rs`: yield-storm, ping-pong-pairs, spawn-storm, - sweeping scheduler count; variant baked in by feature. -- [x] `scripts/bench_rq.sh`: rebuilds per `rq-*` feature, aggregates RQCSV - lines into `bench_results/summary.csv`. Knobs via SMARM_BENCH_* env; - e.g. `SMARM_BENCH_THREADS="1 2 4 8 16 20" ./scripts/bench_rq.sh`. -- [x] Harness validated end-to-end at smoke scale on the 1-core sandbox; - contention curves and the actual variant decision come from the - 20-core box. - -## Phase 5 — Safety hardening & model checking ✅ DONE -- [x] State machine extracted to `src/slot_state.rs` (mechanism only; the - protocol rationale stays in `runtime.rs`) so loom checks the PRODUCTION - transitions. Models: lost-wakeup (park vs unpark), at-most-once - (two unparkers), ABA (stale unpark vs reclaim+reuse), claim coalescing. -- [x] Loom on the rings via `src/sync_shim.rs` (std normally, `loom::sync` - under `--cfg loom`): exactly-once through lap wraparound, push/pop - race, striped two-producer drain. `[target.'cfg(loom)'.dependencies]`; - run with `RUSTFLAGS="--cfg loom" cargo test --lib --release`. - RawMutex is deliberately NOT loom-modeled: futexes can't be, and it is - Drepper's textbook mutex3 with stress + unwind tests. -- [x] NoPreempt / no-unwind / TLS-guard audit: with_runtime + RawMutex guards - + run-queue ops + trace::record all gate preemption (and thereby the - stop sentinel) for their span. Sole remaining exception: channel's std - MutexGuard — the documented first fast-follow. -- [x] Invariant-assertion sweep (the fast-follow, pulled forward): every - StateWord transition self-asserts its precondition; enqueue asserts - exactly (gen, Queued); live_actors underflow asserted; RawMutex - enforces the leaf rule with a debug-build held-count; slab overflow and - ring overflow stay loud panics. House style from here on. - ---- - -## Fast follow (post-v0.5, written down so it isn't lost) -- **Assert the invariants we lean on.** This cycle accumulated load-bearing - invariants: at-most-once-enqueued, queue-ops-under-NoPreempt, never two - cold locks, cold-path generation re-verify under the lock, finalize's - decrement-last, pushes-pair-with-Queued-transitions, thread-local guards - never crossing a switch point. Whenever code RELIES on one and a cheap - check exists, assert it at the point of reliance — `debug_assert!` on hot - paths, full `assert!`/loud panic on cold ones — so a violation fails at - the breakage site, not three modules downstream (the slab-overflow panic - and the queue-op preemption debug_assert are the pattern). Sweep the - existing code for missed spots; new code adopts it as house style. The - phase-5 audit is the natural vehicle for the sweep. -- **Channel mutex migration.** `channel::Inner` is `Arc>` of the - same poison class as the old shared lock; `recv_match` even runs a user - predicate under it. The Phase-1 `check_cancelled` gating already removes the - unwind *source* globally, so channels are poison-safe today — but phase 2 - surfaced a second, sharper reason to migrate: the std `MutexGuard` is held - with preemption ENABLED, so a timeslice switch inside a channel critical - section migrates the actor and unlocks the pthread mutex from a different - OS thread — technically UB (Linux futexes happen to tolerate it). The - `RawMutex` guard disables preemption and is cross-thread-release sound by - construction. Migrate as the first post-v0.5 change. - -## Deferred / explicitly out of scope for v0.5 -- **Unbounded / configurable-bounded actor count.** v0.5 ships a fixed slab - with a loud assert. Revisit with a segmented slab (array of - `AtomicPtr`, doubling segment sizes, append-only) once we actually - hit the cap or a workload demands it. -- **Idle-wakeup eventcount.** Idle scheduler threads keep the current 100µs - poll-sleep. A futex-based eventcount is a later optimization if benches show - idle latency matters. -- **User-facing safe data structures** (ArcSwap-style cells, structurally- - shared persistent maps). Context for the runtime work, not in scope here. -- **IO fd hygiene on actor death** (pre-existing v0.2 TODO in `io.rs`). diff --git a/task.md b/task.md deleted file mode 100644 index 15ff11c..0000000 --- a/task.md +++ /dev/null @@ -1,263 +0,0 @@ -# smarm — task.md (next steps) - -Handoff for a future session reusing this sandbox. Read top to bottom once -before starting; the gotchas section is hard-won and will save you a faceplant. - -## Resume the environment - -- Repo: `smarm`. Two branches (the old single `arm-port` stack was split): - - `master` — the mainline, and HEAD. Carries roadmap #1–#5: cooperative - cancellation, supervisor strategies (one_for_one/all, rest_for_one) + the - orphaned-timer shutdown fix, links/trap_exit, selective receive, gen_server, - and demonitor/`MonitorId`. Tagged `v0.4.0`. x86-64 Linux only. - - `arm-port` — `master` plus a single commit: `feat(arch): aarch64 context - switch + cycle counter`. Extracts the x86-64 context-switch / stack-init / - cycle-counter out of `context.rs` into a `target_arch`-gated `src/arch/` - (x86_64 + aarch64 backends) and adds an AAPCS64 backend. ⚠️ UNTESTED: never - built or run on real ARM hardware. The x86-64 path is unchanged - (`arch/x86_64.rs` is the old `context.rs` body verbatim), so the x86 suite - passing says nothing about the aarch64 backend. Build + test on-device - before trusting it. -- Toolchain is installed but NOT on PATH in a fresh shell. First line of every - session: `. "$HOME/.cargo/env"` (rustc/cargo 1.96). -- Build `cargo build` · all tests `cargo test` · one suite `cargo test --test monitor`. -- Bench probe `cargo bench --bench general` (custom print-only harness; compiles - tokio in release the first time — slow — but `target/` persists across git - checkouts so it's paid once). -- Perf regression check: `git checkout ` → - `cargo bench --bench general | tee before.txt` → `git checkout arm-port` → - run again → diff the `smarm 1-thread` medians for `chained_spawn` and - `yield_many` (those exercise spawn/finalize/scheduler). Numbers are noisy on - this shared CPU; treat as "regression beyond noise?" not a precise delta. - -## Roadmap (dependency order) - -### 1. Cooperative cancellation — the keystone ✅ DONE (`a8ddb4a`) -Everything below (one_for_all/rest_for_one, links) needs a *safe* way to stop a -running peer. Forcible teardown of another green thread's stack is unsound here -(shared heap + Drop). So: cooperative stop the actor observes and unwinds itself. - -Shipped as designed (sentinel unwind, not Result-threading). Notes for what -came next / future readers: -- Stop flag lives on `Actor` behind `Arc` (fresh per spawn), NOT a - `Slot` field — sidesteps the three-place reset, at the cost of one small - alloc per spawn. The scheduler hands the resume path a raw `*const AtomicBool` - (no per-resume refcount traffic); `yield_many` bench stayed at baseline, - `chained_spawn` ~+6% from that alloc (left as-is; move to a `Slot` field if it - ever matters). -- Observation points: amortised `maybe_preempt`/`check!()` path + the wakeup - side of `park_current`/`yield_now`. Sentinel = `StopSentinel` (zero-size), - recognised in the trampoline → `Outcome::Stopped`. `join()` on a stopped actor - returns `Ok(())` (no payload to propagate; reason is on the monitor channel). -- Documented gaps confirmed by tests: no-observation-point loop can't be stopped - (same as preemption); a user `catch_unwind` can swallow the sentinel but the - flag stays set so the next yield re-raises. - - Original plan, for reference: -- Add a per-actor stop flag (Slot field + atomic, or check via shared state). -- `request_stop(pid)`: set the flag, unpark if parked. -- Realize the stop as a **controlled unwind**: when the scheduler resumes a - stop-requested actor, inject a sentinel panic (dedicated payload type) so the - existing `trampoline` `catch_unwind` tears the stack down and runs Drop. The - trampoline recognizes the sentinel and reports a new `Outcome::Stopped` - (distinct from a user `Panic`). This avoids changing every blocking-op - signature. - - Alternative considered: thread `Result<_, Cancelled>` through recv/sleep/ - lock/io. Rejected — large API churn. Go with the sentinel unwind. - - Caveat to document: user code with its own `catch_unwind` can swallow the - sentinel (cf. Erlang `catch`); re-check the flag at the next yield and/or - re-raise. And a tight no-alloc loop without `check!()` can't be stopped — - same inherent limitation as preemption. -- Observation points: `maybe_preempt()`/`check!()` (cheap flag check) and the - blocking parks (recv/sleep/mutex/io) on the stop-driven unpark. -- Tests: looping actor on `check!()` gets stopped → `Outcome::Stopped`, Drop - guards ran; parked-on-recv actor gets stopped; no-check loop documents the gap. - -### 2. one_for_all / rest_for_one + ordered shutdown ✅ DONE (`351dc9c`) -Shipped. What landed vs the plan: -- `Strategy::{OneForOne,OneForAll,RestForOne}` selected via `.strategy()`, - default `OneForOne`. The struct keeps the `OneForOne` name (compat; existing - tests untouched) — a rename to `Supervisor` is a deferred refactor. -- The *triggering* child's `Restart` policy decides whether anything restarts; - the strategy decides which live siblings are cycled (all / index-> after the - failed one). Survivors are `request_stop`'d in reverse start order, awaited on - the existing `supervisor_channel` funnel (no new channel, no `select`), - restarted in start order. One failure = one intensity tick regardless of group - size. Out-of-band signals during an await are stashed and replayed. -- `Signal::Stopped(pid)` + `DownReason::Stopped` added (kept distinct from Exit, - as planned). A `Stopped` signal counts as abnormal for the restart decision. -- Ordered shutdown: on cap-trip / mailbox-close-with-survivors, stop remaining - children in reverse start order and await them (no-op on the normal exit). -- ⚠️ Surfaced + fixed a latent keystone bug (`e80334b`): a cancelled - sleeping/timeout actor orphans its timer entry, and the scheduler's shutdown - check counted pending timers → `run()` hung until the dead actor's deadline - fired (a `sleep(30s)` sleeper hung shutdown 30s). Fix: timers no longer gate - shutdown (`live == 0` already implies nothing a timer could wake); heap is - cleared on exit. Independent of the supervisor work. -- Tests: all-restart (sibling cycled despite clean exit), suffix-restart - (prefix child left alone), reverse-order teardown. - - Original plan, for reference: -- one_for_all: on any child failure, `request_stop` all siblings, await their - termination signals, restart all per spec. -- rest_for_one: stop+restart the failed child and those started after it. -- Supervisor shutdown: stop children in reverse start order. -- Decide signal surface: add `Signal::Stopped(pid)` + `DownReason::Stopped` - rather than folding into Exit (clearer for the supervisor's await logic). -- Tests: all-restart, suffix-restart, reverse-order shutdown. - -### 3. Links + trap_exit ✅ DONE (`6581484`) -- `Slot.links: Vec` (bidirectional); `link`/`unlink`; `trap_exit()` flag - lives on `Actor` (fresh per spawn → a restarted child starts un-trapped, and - no fourth slot-reset site). -- On finalize, reverse links are cleared under the lock (always — keeps the - cascade acyclic), then for each linked peer: abnormal death (`Panic`/ - `Stopped`) → `request_stop(peer)` unless peer traps, in which case deliver an - `ExitSignal` *message* instead. Normal exit does NOT propagate. Linking an - already-dead pid delivers an immediate `NoProc` signal (message if trapping, - else `request_stop(self)` — not a silent no-op). -- Resolved: the trap inbox is a **dedicated** channel (`trap_exit() -> - Receiver`), distinct from the monitor `Down` channel; `ExitSignal` - reuses `DownReason` and carries no panic payload (joiner-only, as with - monitors). `spawn_link` deferred to #5. -- Tests (`tests/link.rs`): linked pair one panics → other stopped (+Drop ran); - trap_exit → other gets a message and survives; normal exit doesn't propagate; - dead-pid link stops a non-trapper / messages a trapper; `unlink` prevents - propagation. - -### 4. Selective receive (independent track) ✅ DONE (`03f3875`) -Shipped. What landed vs the plan: -- `Receiver::recv_match(pred) -> Result` scans the queued - `VecDeque` front-to-back, removes+returns the first match, leaves the rest in - arrival order; parks and re-scans when nothing matches. `try_recv_match` (the - non-blocking variant, mirroring `try_recv`) rolled in same commit. -- Wakeup turned out cheaper than feared: `Sender::send` *already* took - `parked_receiver` on every push, so "wake on ANY send" needed no send-side - change. The one real edit was relaxing `Sender::drop` to unpark the parked - receiver on the last-sender drop regardless of queue emptiness — a selective - receiver can park on a non-empty no-match queue and must wake to observe - closure. No-op for plain `recv` (only ever parks on an empty queue); stress - suite stays green. -- `pred` is `Fn(&T) -> bool` (not `FnMut`) on purpose: it's re-run from scratch - on every scan, so a stateful predicate would re-count surprisingly. It runs - under the channel lock — keep it cheap/pure, don't re-enter the channel. -- Close semantics: `recv_match` returns `Err(RecvError)` only when closed AND no - queued message matches; a match is still returned on a closed channel. - Non-matches are left for a later `recv`. -- Tests (`tests/selective_recv.rs`): out-of-order match pulled first; non-matches - remain in order; park-on-non-empty then wake on a match; closed-with-only- - non-matches → Err; closed-but-match-present → match; `try_recv_match` states. - - Original plan, for reference: -- Add `Receiver::recv_match(pred) -> T`: scan the queued `VecDeque`, remove+return - first match, leave the rest in order; park and re-scan on new arrivals. -- This changes channel wakeup: a parked selective receiver must wake on ANY send - (not just empty→nonempty) and re-scan. Touches `channel.rs` carefully — the - stress tests guard lost-wakeup invariants; keep them green. -- Tests: messages arrive out of interest-order; match pulled first; non-matches - remain for a later `recv`. - -### 5. Grab-bag (each its own small commit) -- `spawn_link`: spawn-and-link atomically (deferred from #3); thin wrapper over - `spawn_under` + `link`, but do it under one lock so there's no window where - the child dies before the link is recorded. -- `demonitor`: needs a per-monitor id to remove a specific sender. Decide the - monitor API NOW before more code depends on it — likely return a - `Monitor { id, rx }` instead of a bare `Receiver`. - ✅ DONE (this commit). What landed vs the plan: - - `monitor()` now returns `Monitor { id, target, rx }` (added `target` over - the sketched `{id, rx}` so `demonitor` jumps straight to the slot instead of - scanning every slot for the id). `MonitorId(u64)` is opaque, from a - monotonic `next_monitor_id` counter on `SharedState`, bumped under the - shared lock in `monitor()` — no atomics, deterministic, never reused. - - `Slot.monitors: Vec<(MonitorId, Sender)>`. The three slot-reset sites - were untouched — they `.clear()`/`Vec::new()`, which is element-type- - agnostic, so no new reset obligation. `finalize_actor` just destructures - `(_, m)` and sends as before. - - `demonitor(&Monitor) -> Option`: `Some(id)` when a live - registration was found+removed, `None` when it had already fired (drained on - finalize), was `NoProc`, or the slot was reclaimed. Chose `Option` - over a bare bool — names which registration went. Generation half of the pid - makes a stale demonitor a clean no-op: a recycled slot index fails - `slot_mut`'s generation check, so it can never strip a different actor's - monitor. - - ⚠️ Reentrancy: the removed `Sender` is `remove`d out of the Vec under the - lock but **dropped after the lock is released** — `Sender::drop` can unpark a - parked receiver → `with_shared`, and the shared mutex is non-reentrant. Same - discipline as `finalize_actor`. - - "Flush" (discard a `Down` the target already queued) falls out of dropping - the `Monitor`: `demonitor(&m); drop(m)`. That's the cleanup the still-to-come - gen_server **call timeout** wants — monitor the server, wait reply-or-Down- - or-deadline, then demonitor+drop so a timed-out call leaks no registration - and no stale `Down`. - - Perf: touches `Slot` + `finalize_actor`, but `chained_spawn`/`yield_many` - register no monitors, so the Vec stays empty (take-empty is identical cost, - finalize loop runs zero times). before/after `general` probe medians within - noise. Tests (`tests/monitor.rs`): demonitor-stops-delivery, one-of-many - (siblings untouched), after-fire-is-None. -- Named registry: `register(name,pid)`/`whereis`/`send_by_name`; a - `HashMap` in `SharedState`. -- gen_server-style call/cast: request-reply correlation as a thin layer over - channels (`call` sends `{req, reply_tx}`, awaits `reply_rx`); no runtime change. - ✅ DONE (`a4fcf6c`). What landed vs the plan: - - `GenServer` trait on the state value: assoc `Call`/`Reply`/`Cast` types, - required `handle_call`/`handle_cast`, optional `init`/`terminate` hooks. - `ServerRef` is a clonable inbox sender + `pid()`; `start` / `start_under`. - - One inbox, not two: a single `Envelope { Call(req, reply_tx) | Cast }` - channel, dispatched by variant. Forced by no-`select`/no-unified-mailbox — - a server can't wait on a call channel and a cast channel at once. - - Server-down falls out of channel closure (no monitor needed): `send` fails - if the inbox is gone; the reply sender drops on the server's unwind so a - parked caller wakes to `Err`. Both → `Call/CastError::ServerDown`. - - `terminate` runs via a drop guard → fires on *every* exit path (clean inbox - close, handler panic, `request_stop`), not just the clean one. Caveat: it - may run mid-unwind, so keep it non-blocking (a panic inside it during an - unwind double-panics → abort). - - No `handle_info`, no call timeout — both deferred to land with timeouts - (`handle_info` needs the still-unmade cross-channel mailbox merge; a call - timeout needs a per-`recv` deadline / `Signal::Timeout`). - - Pure additive layer (no Slot/scheduler/spawn/finalize change) → no perf - check run. Tests (`tests/gen_server.rs`): cast→call roundtrip, init/terminate - ordering, both server-down paths. -- Docs: README now points at the experimental, untested aarch64 port on the - `arm-port` branch. The module table still calls `context` x86-64-only — true - for `master`, since the `src/arch/` split rides on `arm-port`. Fold the arch/ - split and ARM64-supported wording into the README module table + build section - once `arm-port` is validated on hardware and merged. - -## Gotchas / invariants (respect these) - -- **Shared mutex is non-reentrant.** `Sender::send` can call `unpark` → - `with_shared`. NEVER send on a channel while holding the shared lock. Pattern: - `mem::take` the senders/data under the lock, send after releasing. See - `finalize_actor` (supervisor signal + monitor Downs both sent post-lock). -- **`finalize_actor` order:** take stack/waiters/monitors under lock + set - Done/outcome → recycle stack (post-lock) → deliver supervisor Signal + monitor - Downs (post-lock) → unpark joiners → reclaim slot iff `outstanding_handles==0`. - Death notifications always precede reclamation, so a pid carried in a - Signal/Down is still matchable even as its slot is about to be reused. -- **Slot lifecycle is reset in THREE places** — `Slot::vacant()`, - `reclaim_slot()` (runtime.rs), and the slot-init block in `spawn_under` - (scheduler.rs). Any new Slot field must be reset in all three (monitors was). -- **Pid = (index, generation);** stale handles caught by generation mismatch in - `slot()/slot_mut()`. The monitor `NoProc` path relies on this. -- **No `select`, no unified per-process mailbox.** Why the supervisor uses the - single `supervisor_channel` funnel rather than N monitor channels. trap_exit - resolved this by giving each trapping actor a dedicated `Receiver` - inbox (see #3); selective receive (#4) stayed *per-channel* (`recv_match` - scans one channel's queue) rather than introducing a cross-channel mailbox — - if selective receive ever needs to span the monitor/trap inboxes too, that - cross-channel merge is the still-unmade decision. -- **Cooperative-only**: preemption and (future) cancellation both depend on the - actor reaching `check!()`/yield/alloc/blocking points. -- `run()` is single-thread (`Config::exact(1)`); tests rely on deterministic - single-thread ordering (parent runs until it parks). Multi-thread via - `runtime::init(Config…)`. - -## Workflow expectations (from the human) - -- TDD: write the failing test first, then implement. -- Commit incrementally with conventional-commit messages; keep each commit a - reviewable unit (they diff in their IDE and are the filter to the codebase). -- Run the full suite before each commit; check perf when a change touches - Slot/scheduler/spawn/finalize hot paths.