docs(roadmap): consolidate task.md + ROADMAP_v0.5.md into ROADMAP.md

Single source of truth: v0.4 history, v0.5 phases, v0.6 open tracks, and
invariants/gotchas. Old task.md and ROADMAP_v0.5.md removed.
This commit is contained in:
smarm
2026-06-09 21:53:01 +00:00
parent a7ca6646d7
commit 5428697f9e
3 changed files with 186 additions and 420 deletions
+186
View File
@@ -0,0 +1,186 @@
# smarm — Roadmap
## v0.4 — Actor primitives ✅ DONE
### 1. Cooperative cancellation ✅ (`a8ddb4a`)
Stop flag on `Actor` behind `Arc<AtomicBool>`. Observation points:
`maybe_preempt`/`check!()` + the wakeup side of `park_current`/`yield_now`.
Sentinel = `StopSentinel` (zero-size), caught in trampoline → `Outcome::Stopped`.
`join()` on a stopped actor returns `Ok(())`.
Known gap (by design): a tight no-alloc loop without `check!()` cannot be
stopped, same as preemption.
### 2. one_for_all / rest_for_one + ordered shutdown ✅ (`351dc9c`)
`Strategy::{OneForOne,OneForAll,RestForOne}` via `.strategy()`. Triggering
child's `Restart` policy decides whether anything restarts; strategy decides
which siblings are cycled. Survivors stopped in reverse start order, awaited on
the existing `supervisor_channel` funnel, restarted in start order.
`Signal::Stopped(pid)` + `DownReason::Stopped` added (distinct from Exit).
Also fixed a latent bug (`e80334b`): orphaned timers were gating shutdown.
`live == 0` is now the shutdown condition; timer heap cleared on exit.
### 3. Links + trap_exit ✅ (`6581484`)
`Slot.links: Vec<Pid>` (bidirectional). On finalize: abnormal death →
`request_stop(peer)` unless peer traps, in which case deliver `ExitSignal`
message. Normal exit does not propagate. `trap_exit()` returns a dedicated
`Receiver<ExitSignal>` (distinct from monitor `Down` channel).
### 4. Selective receive ✅ (`03f3875`)
`Receiver::recv_match(pred)` + `try_recv_match`. Scans `VecDeque` front-to-back,
removes+returns first match, parks and re-scans on new arrivals. `pred` is
`Fn(&T) -> bool` (not `FnMut`; runs under the channel lock — keep it cheap).
### 5. Grab-bag ✅
- **demonitor** (`5181037`): `monitor()``Monitor { id, target, rx }`.
`demonitor(&m) -> Option<MonitorId>`. Dropped `Sender` released post-lock
(non-reentrant shared mutex discipline).
- **gen_server** (`55221e9`): `GenServer` trait, `call`/`cast` over one
`Envelope` inbox. `terminate` hook via drop guard. No `handle_info`, no call
timeout (deferred — needs cross-channel mailbox merge or per-recv deadline).
- **Named registry**: `register`/`whereis`/`send_by_name`**not yet done**.
HashMap in `SharedState`. No dependencies; can land any time.
- **arm-port branch**: aarch64 context-switch backend extracted into
`src/arch/`. Untested on hardware. Merge only after on-device validation.
---
## v0.5 — Runtime decomposition & run-path scaling ✅ DONE
Goal: dismantle the single `Mutex<SharedState>` along the run path so the
runtime scales to dozens of cores, with crash isolation hardened so a torn
write or a stray cancellation can never poison shared runtime state.
Guiding rules established this cycle:
- **Lock count that matters is hot-path locks.** Cold lifecycle paths may keep a
small per-slot lock; the run path targets zero locks.
- **Lock ordering: io-before-shared, slot locks are leaves** (never hold two
slot locks at once).
- **No unwind inside a runtime critical section.** Stop sentinel only fires at
lock-free observation points.
- **Never hold a thread-local guard across a possible switch point.** An actor
can be preempted at any allocation and resume on a different OS thread.
`with_runtime`, `with_shared`, and every `RawMutex` guard disable preemption
for this reason.
### Phase 1 — State decomposition ✅ (`3c7e26b`)
`next_monitor_id``AtomicU64`. `timers` → own `Mutex<Timers>`. `io` → own
`Mutex<Option<IoThread>>`. `pending_closures` → folded into `Slot::pending_closure`.
`check_cancelled()` gated behind `PREEMPTION_ENABLED` (poison fix).
`SharedState` now holds only: `slots`, `free_list`, `run_queue`, `root_pid`.
### Phase 2 — Slot table split ✅ (`5e0c9d4`)
Fixed slab `Box<[Slot]>`. Generation packed into state word: one
`AtomicU64 = (gen << 32) | state` — gen check is atomic with every transition.
Per-slot CAS state machine `Vacant/Queued/Running/RunningNotified/Parked/Done`.
Per-slot `RawMutex` for cold collections. Free list → `RawMutex<Vec<u32>>`.
`live_actors` atomic; termination = io_out==0 && queue empty && live==0.
### Phase 3 — Pluggable run queue ✅ (`1b3b618`)
Compile-time selected via feature flags. Variants:
- `rq-mutex``Mutex<VecDeque>`, baseline.
- `rq-mpmc` — hand-rolled Vyukov bounded MPMC ring.
- `rq-striped` — M Vyukov rings, fetch-add ticket distribution.
All variants compile in every build; feature only picks the alias. Bounded rings
are sound via slab cap + at-most-once-enqueued invariant.
Queue ops require preemption disabled (a producer suspended mid-publish stalls
every consumer behind its cell — livelock).
### Phase 4 — Bench harness ✅ (`6d9f369`)
`benches/rq_micro.rs`: raw structure sweep. `benches/rq_runtime.rs`:
yield-storm, ping-pong-pairs, spawn-storm. `scripts/bench_rq.sh`: rebuilds per
feature, aggregates to `bench_results/summary.csv`.
### Phase 5 — Safety hardening & model checking ✅ (`039703d`)
State machine extracted to `src/slot_state.rs` for loom checking. Loom on the
rings via `src/sync_shim.rs`. `NoPreempt`/no-unwind/TLS-guard audit complete.
Invariant-assertion sweep: every `StateWord` transition self-asserts its
precondition.
---
## v0.6 — Fast follows & open tracks
Priority order within the cycle; each item is its own commit.
### 1. Channel mutex migration (first change)
`channel::Inner<T>` holds a `std::sync::Mutex`. Phase 2 surfaced a sharp
correctness issue: the `MutexGuard` is held with preemption **enabled**, so a
timeslice switch inside a channel critical section migrates the actor and
releases the pthread mutex from a different OS thread — UB (Linux futexes happen
to tolerate it, but it's not guaranteed). The Phase-1 `check_cancelled` gating
already removes the unwind source, so poison is safe today; this is about the
cross-thread release.
Migrate to `RawMutex` (which disables preemption and is cross-thread-release
sound by construction). `recv_match` runs a user predicate under this lock —
keep it cheap/pure.
### 2. Named registry
`register(name, pid)` / `whereis(name) -> Option<Pid>` / `send_by_name`.
`HashMap<String, Pid>` in `SharedState`, guarded by the shared lock. No
architecture changes; can land immediately after #1.
### 3. gen_server: handle_info + call timeout
- `handle_info`: needs a cross-channel mailbox merge (call/cast inbox +
out-of-band signals). The still-unmade decision: whether to introduce a
unified per-actor mailbox or keep per-channel `recv_match` composition.
- Call timeout: `call` with a deadline — monitor the server, wait
reply-or-Down-or-deadline, then `demonitor`+drop so a timed-out call leaks
no registration. Requires a per-`recv` deadline / `Signal::Timeout` primitive
that doesn't exist yet.
These two can land independently; call timeout is smaller and more useful
first.
### 4. Unbounded / configurable-bounded actor count
v0.5 ships a fixed slab with a loud assert (`Config::max_actors(n)`, default
16 384). Revisit with a segmented slab (array of `AtomicPtr<Segment>`, doubling
segment sizes, append-only) once the cap is actually hit or a workload demands
it. Do not let the fixed cap calcify into an assumption.
### 5. Idle-wakeup eventcount
Idle scheduler threads currently poll-sleep at 100 µs. A futex-based eventcount
would reduce idle latency. Defer until benches show it matters.
### 6. IO fd hygiene on actor death
Pre-existing v0.2 TODO in `io.rs`: fds registered with epoll for a dead actor
are not cleaned up. Audit and fix.
### 7. arm-port validation & merge
`arm-port` branch carries an AAPCS64 context-switch backend, never run on
hardware. Steps: build + run full test suite on an aarch64 device; check
`chained_spawn` / `yield_many` bench medians for regressions vs x86 baseline;
merge and update README module table to reflect multi-arch support.
---
## Invariants & gotchas (respect these across all cycles)
- **Shared mutex is non-reentrant.** `Sender::send` can call `unpark`
`with_shared`. Never send on a channel while holding the shared lock. Pattern:
`mem::take` data under the lock, send after releasing. See `finalize_actor`.
- **`finalize_actor` order:** take stack/waiters/monitors under lock + set
Done/outcome → recycle stack → deliver supervisor Signal + monitor Downs →
unpark joiners → reclaim slot if `outstanding_handles==0`. Death notifications
always precede reclamation.
- **Slot lifecycle reset in THREE places:** `Slot::vacant()`, `reclaim_slot()`
(runtime.rs), slot-init block in `spawn_under` (scheduler.rs). Any new `Slot`
field must be reset in all three.
- **Pid = (index, generation).** Stale handles caught by generation mismatch in
`slot()/slot_mut()`. The monitor `NoProc` path relies on this.
- **No `select`, no unified per-process mailbox.** Why the supervisor uses the
single `supervisor_channel` funnel. `recv_match` is per-channel only; a
cross-channel merge is the still-unmade decision (see v0.6 #3).
- **Cooperative-only.** Preemption and cancellation both depend on the actor
reaching `check!()`/yield/alloc/blocking points.
- **Queue ops require preemption disabled.** A producer suspended mid-publish
stalls every consumer — livelock. `with_runtime`, `with_shared`, and
`RawMutex` guards all disable preemption for their span.
- **`run()` is single-thread** (`Config::exact(1)`); tests rely on deterministic
single-thread ordering. Multi-thread via `runtime::init(Config…)`.
-157
View File
@@ -1,157 +0,0 @@
# smarm v0.5 — Runtime decomposition & run-path scaling
Goal: dismantle the single `Mutex<SharedState>` along the run path so the
runtime scales to dozens of cores, with crash isolation hardened so a torn
write or a stray cancellation can never poison shared runtime state.
Guiding rules established this cycle:
- **Lock count that matters is hot-path locks.** Cold lifecycle paths
(spawn/join/monitor/link/finalize) may keep a small per-slot lock; the run
path (yield/park/unpark/pop/resume) targets zero locks.
- **Lock ordering: io-before-shared, and slot locks are leaves** (never hold
two slot locks at once). Any new lock states its position in this order.
- **No unwind inside a runtime critical section.** The stop sentinel only
fires at lock-free observation points.
- **Never hold a thread-local guard across a possible switch point.** An
actor can be preempted at any allocation and resume on a different OS
thread; a live `RefCell` borrow (or std `MutexGuard`) then pairs its
acquire/release across two threads' locals — count underflow / UB.
`with_runtime`, `with_shared`, and every `RawMutex` guard disable
preemption for exactly this reason (phase 2 found this the hard way).
---
## Phase 1 — State decomposition: easy peel-offs ✅ DONE
Split independent concerns out of `SharedState` so the global lock guards less.
- [x] `next_monitor_id``AtomicU64` on `RuntimeInner` (lock-free id minting).
- [x] `timers` → own `Mutex<Timers>` (only drain-winner + blocking prims touch it).
- [x] `io` → own `Mutex<Option<IoThread>>` (completion queue already self-locked).
- [x] `pending_closures: Vec` → folded into `Slot::pending_closure` (per-actor data).
- [x] Termination check split: read io liveness *before* `shared`; ordering
argument documented in `schedule_loop`.
- [x] **Poison fix:** gate `check_cancelled()` in `maybe_preempt` behind
`PREEMPTION_ENABLED`, so a cancellation sentinel can never unwind while a
`std::sync::Mutex` (shared/channel) is held. Regression test:
`tests/poison_stop.rs`.
`SharedState` now holds only: `slots`, `free_list`, `run_queue`, `root_pid`.
## Phase 2 — Slot table split ✅ DONE
Make slot lookup lock-free and per-slot state independently mutable.
- [x] Fixed slab: `Box<[Slot]>`, slots never move → stable
addresses, lock-free index. **Assert on exhaustion** with a panic message
naming `Config::max_actors(n)` as the fix. Default `max_actors = 16_384`.
(Mental note: must become unbounded or configurable-bounded later — see
"Deferred" below. Do not let the fixed cap calcify into an assumption.)
- [x] Generation packed INTO the state word (better than the planned separate
`AtomicU32`): one `AtomicU64 = (gen << 32) | state`, so the gen check is
atomic with every transition — no ABA, no spurious unparks, by CAS.
- [x] Per-slot CAS state machine `Vacant/Queued/Running/RunningNotified/
Parked/Done` replacing `state` + `pending_unpark`. Also fixed a latent
lost wakeup in the Blocking-IO completion path (result set for a
still-Running actor without a flag).
- [x] `sp` → relaxed `AtomicUsize`; stop flag + first-resume closure as
`AtomicPtr`s — resume path fully lock-free.
- [x] Per-slot raw non-poisoning futex mutex (`src/raw_mutex.rs`) for the
cold collections. Guard enters `NoPreempt`.
- [x] Free list → `RawMutex<Vec<u32>>` (a leaf lock). Treiber/striped only if
the phase-4 spawn-storm bench shows contention — revisit then.
- [x] `finalize_actor` link cascade locks peers one at a time; acyclicity
argument written at the site. `link()` registers target-first with the
race argument at the site.
- [x] `live_actors` atomic; termination = io_out == 0 (read pre-queue-lock)
&& queue empty && live == 0; decrement-last ordering documented as the
correctness crux at the site.
## Phase 3 — Pluggable run queue ✅ DONE (shootout = phase 4 harness)
- [x] `RunQueue` alias in `src/run_queue.rs`, compile-time selected,
`compile_error!` guards for zero / >1 features. No runtime dispatch.
All variants compile in every build (unit tests always run); the
feature only picks the alias. Non-default variants:
`--no-default-features --features rq-…`.
- `rq-mutex` — `Mutex<VecDeque>`, the control/baseline.
- `rq-mpmc` — single hand-rolled Vyukov bounded MPMC ring
(per-cell seq numbers). Strict FIFO; "one hot cache line".
- `rq-striped` — M Vyukov rings, fetch-add ticket distribution. Relaxed
FIFO, reordering bounded by ~M. Predicted winner @20c.
- [x] Bounded rings sound via slab cap + at-most-once-enqueued (occupancy ≤
`max_actors`); mpmc capacity = next_pow2(max_actors), striped
Σcapacity ≈ 2×max_actors with probe-from-home-stripe. A full ring
panics as an invariant violation (double enqueue), never spins.
- [x] All hand-rolled, dependency-free.
- [x] Two contracts surfaced by the extraction, documented + debug-asserted
in `run_queue.rs`: queue ops require preemption disabled (a producer
suspended mid-publish stalls every consumer behind its cell — livelock);
and pop-None is a snapshot, not a fence — termination is counter-first
(`live == 0` alone implies the queue holds nothing actionable; argument
rewritten at the `schedule_loop` site).
## Phase 4 — Bench harness ✅ DONE (harness; real numbers from the 20-core box)
- [x] `benches/rq_micro.rs`: raw structures, threads × p:c ratio sweep. One
binary covers all three structures (types compile in every build).
- [x] `benches/rq_runtime.rs`: yield-storm, ping-pong-pairs, spawn-storm,
sweeping scheduler count; variant baked in by feature.
- [x] `scripts/bench_rq.sh`: rebuilds per `rq-*` feature, aggregates RQCSV
lines into `bench_results/summary.csv`. Knobs via SMARM_BENCH_* env;
e.g. `SMARM_BENCH_THREADS="1 2 4 8 16 20" ./scripts/bench_rq.sh`.
- [x] Harness validated end-to-end at smoke scale on the 1-core sandbox;
contention curves and the actual variant decision come from the
20-core box.
## Phase 5 — Safety hardening & model checking ✅ DONE
- [x] State machine extracted to `src/slot_state.rs` (mechanism only; the
protocol rationale stays in `runtime.rs`) so loom checks the PRODUCTION
transitions. Models: lost-wakeup (park vs unpark), at-most-once
(two unparkers), ABA (stale unpark vs reclaim+reuse), claim coalescing.
- [x] Loom on the rings via `src/sync_shim.rs` (std normally, `loom::sync`
under `--cfg loom`): exactly-once through lap wraparound, push/pop
race, striped two-producer drain. `[target.'cfg(loom)'.dependencies]`;
run with `RUSTFLAGS="--cfg loom" cargo test --lib --release`.
RawMutex is deliberately NOT loom-modeled: futexes can't be, and it is
Drepper's textbook mutex3 with stress + unwind tests.
- [x] NoPreempt / no-unwind / TLS-guard audit: with_runtime + RawMutex guards
+ run-queue ops + trace::record all gate preemption (and thereby the
stop sentinel) for their span. Sole remaining exception: channel's std
MutexGuard — the documented first fast-follow.
- [x] Invariant-assertion sweep (the fast-follow, pulled forward): every
StateWord transition self-asserts its precondition; enqueue asserts
exactly (gen, Queued); live_actors underflow asserted; RawMutex
enforces the leaf rule with a debug-build held-count; slab overflow and
ring overflow stay loud panics. House style from here on.
---
## Fast follow (post-v0.5, written down so it isn't lost)
- **Assert the invariants we lean on.** This cycle accumulated load-bearing
invariants: at-most-once-enqueued, queue-ops-under-NoPreempt, never two
cold locks, cold-path generation re-verify under the lock, finalize's
decrement-last, pushes-pair-with-Queued-transitions, thread-local guards
never crossing a switch point. Whenever code RELIES on one and a cheap
check exists, assert it at the point of reliance — `debug_assert!` on hot
paths, full `assert!`/loud panic on cold ones — so a violation fails at
the breakage site, not three modules downstream (the slab-overflow panic
and the queue-op preemption debug_assert are the pattern). Sweep the
existing code for missed spots; new code adopts it as house style. The
phase-5 audit is the natural vehicle for the sweep.
- **Channel mutex migration.** `channel::Inner<T>` is `Arc<Mutex<_>>` of the
same poison class as the old shared lock; `recv_match` even runs a user
predicate under it. The Phase-1 `check_cancelled` gating already removes the
unwind *source* globally, so channels are poison-safe today — but phase 2
surfaced a second, sharper reason to migrate: the std `MutexGuard` is held
with preemption ENABLED, so a timeslice switch inside a channel critical
section migrates the actor and unlocks the pthread mutex from a different
OS thread — technically UB (Linux futexes happen to tolerate it). The
`RawMutex` guard disables preemption and is cross-thread-release sound by
construction. Migrate as the first post-v0.5 change.
## Deferred / explicitly out of scope for v0.5
- **Unbounded / configurable-bounded actor count.** v0.5 ships a fixed slab
with a loud assert. Revisit with a segmented slab (array of
`AtomicPtr<Segment>`, doubling segment sizes, append-only) once we actually
hit the cap or a workload demands it.
- **Idle-wakeup eventcount.** Idle scheduler threads keep the current 100µs
poll-sleep. A futex-based eventcount is a later optimization if benches show
idle latency matters.
- **User-facing safe data structures** (ArcSwap-style cells, structurally-
shared persistent maps). Context for the runtime work, not in scope here.
- **IO fd hygiene on actor death** (pre-existing v0.2 TODO in `io.rs`).
-263
View File
@@ -1,263 +0,0 @@
# smarm — task.md (next steps)
Handoff for a future session reusing this sandbox. Read top to bottom once
before starting; the gotchas section is hard-won and will save you a faceplant.
## Resume the environment
- Repo: `smarm`. Two branches (the old single `arm-port` stack was split):
- `master` — the mainline, and HEAD. Carries roadmap #1#5: cooperative
cancellation, supervisor strategies (one_for_one/all, rest_for_one) + the
orphaned-timer shutdown fix, links/trap_exit, selective receive, gen_server,
and demonitor/`MonitorId`. Tagged `v0.4.0`. x86-64 Linux only.
- `arm-port``master` plus a single commit: `feat(arch): aarch64 context
switch + cycle counter`. Extracts the x86-64 context-switch / stack-init /
cycle-counter out of `context.rs` into a `target_arch`-gated `src/arch/`
(x86_64 + aarch64 backends) and adds an AAPCS64 backend. ⚠️ UNTESTED: never
built or run on real ARM hardware. The x86-64 path is unchanged
(`arch/x86_64.rs` is the old `context.rs` body verbatim), so the x86 suite
passing says nothing about the aarch64 backend. Build + test on-device
before trusting it.
- Toolchain is installed but NOT on PATH in a fresh shell. First line of every
session: `. "$HOME/.cargo/env"` (rustc/cargo 1.96).
- Build `cargo build` · all tests `cargo test` · one suite `cargo test --test monitor`.
- Bench probe `cargo bench --bench general` (custom print-only harness; compiles
tokio in release the first time — slow — but `target/` persists across git
checkouts so it's paid once).
- Perf regression check: `git checkout <pre-change-sha>` →
`cargo bench --bench general | tee before.txt` → `git checkout arm-port` →
run again → diff the `smarm 1-thread` medians for `chained_spawn` and
`yield_many` (those exercise spawn/finalize/scheduler). Numbers are noisy on
this shared CPU; treat as "regression beyond noise?" not a precise delta.
## Roadmap (dependency order)
### 1. Cooperative cancellation — the keystone ✅ DONE (`a8ddb4a`)
Everything below (one_for_all/rest_for_one, links) needs a *safe* way to stop a
running peer. Forcible teardown of another green thread's stack is unsound here
(shared heap + Drop). So: cooperative stop the actor observes and unwinds itself.
Shipped as designed (sentinel unwind, not Result-threading). Notes for what
came next / future readers:
- Stop flag lives on `Actor` behind `Arc<AtomicBool>` (fresh per spawn), NOT a
`Slot` field — sidesteps the three-place reset, at the cost of one small
alloc per spawn. The scheduler hands the resume path a raw `*const AtomicBool`
(no per-resume refcount traffic); `yield_many` bench stayed at baseline,
`chained_spawn` ~+6% from that alloc (left as-is; move to a `Slot` field if it
ever matters).
- Observation points: amortised `maybe_preempt`/`check!()` path + the wakeup
side of `park_current`/`yield_now`. Sentinel = `StopSentinel` (zero-size),
recognised in the trampoline → `Outcome::Stopped`. `join()` on a stopped actor
returns `Ok(())` (no payload to propagate; reason is on the monitor channel).
- Documented gaps confirmed by tests: no-observation-point loop can't be stopped
(same as preemption); a user `catch_unwind` can swallow the sentinel but the
flag stays set so the next yield re-raises.
Original plan, for reference:
- Add a per-actor stop flag (Slot field + atomic, or check via shared state).
- `request_stop(pid)`: set the flag, unpark if parked.
- Realize the stop as a **controlled unwind**: when the scheduler resumes a
stop-requested actor, inject a sentinel panic (dedicated payload type) so the
existing `trampoline` `catch_unwind` tears the stack down and runs Drop. The
trampoline recognizes the sentinel and reports a new `Outcome::Stopped`
(distinct from a user `Panic`). This avoids changing every blocking-op
signature.
- Alternative considered: thread `Result<_, Cancelled>` through recv/sleep/
lock/io. Rejected — large API churn. Go with the sentinel unwind.
- Caveat to document: user code with its own `catch_unwind` can swallow the
sentinel (cf. Erlang `catch`); re-check the flag at the next yield and/or
re-raise. And a tight no-alloc loop without `check!()` can't be stopped —
same inherent limitation as preemption.
- Observation points: `maybe_preempt()`/`check!()` (cheap flag check) and the
blocking parks (recv/sleep/mutex/io) on the stop-driven unpark.
- Tests: looping actor on `check!()` gets stopped → `Outcome::Stopped`, Drop
guards ran; parked-on-recv actor gets stopped; no-check loop documents the gap.
### 2. one_for_all / rest_for_one + ordered shutdown ✅ DONE (`351dc9c`)
Shipped. What landed vs the plan:
- `Strategy::{OneForOne,OneForAll,RestForOne}` selected via `.strategy()`,
default `OneForOne`. The struct keeps the `OneForOne` name (compat; existing
tests untouched) — a rename to `Supervisor` is a deferred refactor.
- The *triggering* child's `Restart` policy decides whether anything restarts;
the strategy decides which live siblings are cycled (all / index-> after the
failed one). Survivors are `request_stop`'d in reverse start order, awaited on
the existing `supervisor_channel` funnel (no new channel, no `select`),
restarted in start order. One failure = one intensity tick regardless of group
size. Out-of-band signals during an await are stashed and replayed.
- `Signal::Stopped(pid)` + `DownReason::Stopped` added (kept distinct from Exit,
as planned). A `Stopped` signal counts as abnormal for the restart decision.
- Ordered shutdown: on cap-trip / mailbox-close-with-survivors, stop remaining
children in reverse start order and await them (no-op on the normal exit).
- ⚠️ Surfaced + fixed a latent keystone bug (`e80334b`): a cancelled
sleeping/timeout actor orphans its timer entry, and the scheduler's shutdown
check counted pending timers → `run()` hung until the dead actor's deadline
fired (a `sleep(30s)` sleeper hung shutdown 30s). Fix: timers no longer gate
shutdown (`live == 0` already implies nothing a timer could wake); heap is
cleared on exit. Independent of the supervisor work.
- Tests: all-restart (sibling cycled despite clean exit), suffix-restart
(prefix child left alone), reverse-order teardown.
Original plan, for reference:
- one_for_all: on any child failure, `request_stop` all siblings, await their
termination signals, restart all per spec.
- rest_for_one: stop+restart the failed child and those started after it.
- Supervisor shutdown: stop children in reverse start order.
- Decide signal surface: add `Signal::Stopped(pid)` + `DownReason::Stopped`
rather than folding into Exit (clearer for the supervisor's await logic).
- Tests: all-restart, suffix-restart, reverse-order shutdown.
### 3. Links + trap_exit ✅ DONE (`6581484`)
- `Slot.links: Vec<Pid>` (bidirectional); `link`/`unlink`; `trap_exit()` flag
lives on `Actor` (fresh per spawn → a restarted child starts un-trapped, and
no fourth slot-reset site).
- On finalize, reverse links are cleared under the lock (always — keeps the
cascade acyclic), then for each linked peer: abnormal death (`Panic`/
`Stopped`) → `request_stop(peer)` unless peer traps, in which case deliver an
`ExitSignal` *message* instead. Normal exit does NOT propagate. Linking an
already-dead pid delivers an immediate `NoProc` signal (message if trapping,
else `request_stop(self)` — not a silent no-op).
- Resolved: the trap inbox is a **dedicated** channel (`trap_exit() ->
Receiver<ExitSignal>`), distinct from the monitor `Down` channel; `ExitSignal`
reuses `DownReason` and carries no panic payload (joiner-only, as with
monitors). `spawn_link` deferred to #5.
- Tests (`tests/link.rs`): linked pair one panics → other stopped (+Drop ran);
trap_exit → other gets a message and survives; normal exit doesn't propagate;
dead-pid link stops a non-trapper / messages a trapper; `unlink` prevents
propagation.
### 4. Selective receive (independent track) ✅ DONE (`03f3875`)
Shipped. What landed vs the plan:
- `Receiver::recv_match(pred) -> Result<T, RecvError>` scans the queued
`VecDeque` front-to-back, removes+returns the first match, leaves the rest in
arrival order; parks and re-scans when nothing matches. `try_recv_match` (the
non-blocking variant, mirroring `try_recv`) rolled in same commit.
- Wakeup turned out cheaper than feared: `Sender::send` *already* took
`parked_receiver` on every push, so "wake on ANY send" needed no send-side
change. The one real edit was relaxing `Sender::drop` to unpark the parked
receiver on the last-sender drop regardless of queue emptiness — a selective
receiver can park on a non-empty no-match queue and must wake to observe
closure. No-op for plain `recv` (only ever parks on an empty queue); stress
suite stays green.
- `pred` is `Fn(&T) -> bool` (not `FnMut`) on purpose: it's re-run from scratch
on every scan, so a stateful predicate would re-count surprisingly. It runs
under the channel lock — keep it cheap/pure, don't re-enter the channel.
- Close semantics: `recv_match` returns `Err(RecvError)` only when closed AND no
queued message matches; a match is still returned on a closed channel.
Non-matches are left for a later `recv`.
- Tests (`tests/selective_recv.rs`): out-of-order match pulled first; non-matches
remain in order; park-on-non-empty then wake on a match; closed-with-only-
non-matches → Err; closed-but-match-present → match; `try_recv_match` states.
Original plan, for reference:
- Add `Receiver::recv_match(pred) -> T`: scan the queued `VecDeque`, remove+return
first match, leave the rest in order; park and re-scan on new arrivals.
- This changes channel wakeup: a parked selective receiver must wake on ANY send
(not just empty→nonempty) and re-scan. Touches `channel.rs` carefully — the
stress tests guard lost-wakeup invariants; keep them green.
- Tests: messages arrive out of interest-order; match pulled first; non-matches
remain for a later `recv`.
### 5. Grab-bag (each its own small commit)
- `spawn_link`: spawn-and-link atomically (deferred from #3); thin wrapper over
`spawn_under` + `link`, but do it under one lock so there's no window where
the child dies before the link is recorded.
- `demonitor`: needs a per-monitor id to remove a specific sender. Decide the
monitor API NOW before more code depends on it — likely return a
`Monitor { id, rx }` instead of a bare `Receiver<Down>`.
✅ DONE (this commit). What landed vs the plan:
- `monitor()` now returns `Monitor { id, target, rx }` (added `target` over
the sketched `{id, rx}` so `demonitor` jumps straight to the slot instead of
scanning every slot for the id). `MonitorId(u64)` is opaque, from a
monotonic `next_monitor_id` counter on `SharedState`, bumped under the
shared lock in `monitor()` — no atomics, deterministic, never reused.
- `Slot.monitors: Vec<(MonitorId, Sender<Down>)>`. The three slot-reset sites
were untouched — they `.clear()`/`Vec::new()`, which is element-type-
agnostic, so no new reset obligation. `finalize_actor` just destructures
`(_, m)` and sends as before.
- `demonitor(&Monitor) -> Option<MonitorId>`: `Some(id)` when a live
registration was found+removed, `None` when it had already fired (drained on
finalize), was `NoProc`, or the slot was reclaimed. Chose `Option<MonitorId>`
over a bare bool — names which registration went. Generation half of the pid
makes a stale demonitor a clean no-op: a recycled slot index fails
`slot_mut`'s generation check, so it can never strip a different actor's
monitor.
- ⚠️ Reentrancy: the removed `Sender` is `remove`d out of the Vec under the
lock but **dropped after the lock is released** — `Sender::drop` can unpark a
parked receiver → `with_shared`, and the shared mutex is non-reentrant. Same
discipline as `finalize_actor`.
- "Flush" (discard a `Down` the target already queued) falls out of dropping
the `Monitor`: `demonitor(&m); drop(m)`. That's the cleanup the still-to-come
gen_server **call timeout** wants — monitor the server, wait reply-or-Down-
or-deadline, then demonitor+drop so a timed-out call leaks no registration
and no stale `Down`.
- Perf: touches `Slot` + `finalize_actor`, but `chained_spawn`/`yield_many`
register no monitors, so the Vec stays empty (take-empty is identical cost,
finalize loop runs zero times). before/after `general` probe medians within
noise. Tests (`tests/monitor.rs`): demonitor-stops-delivery, one-of-many
(siblings untouched), after-fire-is-None.
- Named registry: `register(name,pid)`/`whereis`/`send_by_name`; a
`HashMap<String,Pid>` in `SharedState`.
- gen_server-style call/cast: request-reply correlation as a thin layer over
channels (`call` sends `{req, reply_tx}`, awaits `reply_rx`); no runtime change.
✅ DONE (`a4fcf6c`). What landed vs the plan:
- `GenServer` trait on the state value: assoc `Call`/`Reply`/`Cast` types,
required `handle_call`/`handle_cast`, optional `init`/`terminate` hooks.
`ServerRef<G>` is a clonable inbox sender + `pid()`; `start` / `start_under`.
- One inbox, not two: a single `Envelope { Call(req, reply_tx) | Cast }`
channel, dispatched by variant. Forced by no-`select`/no-unified-mailbox —
a server can't wait on a call channel and a cast channel at once.
- Server-down falls out of channel closure (no monitor needed): `send` fails
if the inbox is gone; the reply sender drops on the server's unwind so a
parked caller wakes to `Err`. Both → `Call/CastError::ServerDown`.
- `terminate` runs via a drop guard → fires on *every* exit path (clean inbox
close, handler panic, `request_stop`), not just the clean one. Caveat: it
may run mid-unwind, so keep it non-blocking (a panic inside it during an
unwind double-panics → abort).
- No `handle_info`, no call timeout — both deferred to land with timeouts
(`handle_info` needs the still-unmade cross-channel mailbox merge; a call
timeout needs a per-`recv` deadline / `Signal::Timeout`).
- Pure additive layer (no Slot/scheduler/spawn/finalize change) → no perf
check run. Tests (`tests/gen_server.rs`): cast→call roundtrip, init/terminate
ordering, both server-down paths.
- Docs: README now points at the experimental, untested aarch64 port on the
`arm-port` branch. The module table still calls `context` x86-64-only — true
for `master`, since the `src/arch/` split rides on `arm-port`. Fold the arch/
split and ARM64-supported wording into the README module table + build section
once `arm-port` is validated on hardware and merged.
## Gotchas / invariants (respect these)
- **Shared mutex is non-reentrant.** `Sender::send` can call `unpark` →
`with_shared`. NEVER send on a channel while holding the shared lock. Pattern:
`mem::take` the senders/data under the lock, send after releasing. See
`finalize_actor` (supervisor signal + monitor Downs both sent post-lock).
- **`finalize_actor` order:** take stack/waiters/monitors under lock + set
Done/outcome → recycle stack (post-lock) → deliver supervisor Signal + monitor
Downs (post-lock) → unpark joiners → reclaim slot iff `outstanding_handles==0`.
Death notifications always precede reclamation, so a pid carried in a
Signal/Down is still matchable even as its slot is about to be reused.
- **Slot lifecycle is reset in THREE places** — `Slot::vacant()`,
`reclaim_slot()` (runtime.rs), and the slot-init block in `spawn_under`
(scheduler.rs). Any new Slot field must be reset in all three (monitors was).
- **Pid = (index, generation);** stale handles caught by generation mismatch in
`slot()/slot_mut()`. The monitor `NoProc` path relies on this.
- **No `select`, no unified per-process mailbox.** Why the supervisor uses the
single `supervisor_channel` funnel rather than N monitor channels. trap_exit
resolved this by giving each trapping actor a dedicated `Receiver<ExitSignal>`
inbox (see #3); selective receive (#4) stayed *per-channel* (`recv_match`
scans one channel's queue) rather than introducing a cross-channel mailbox —
if selective receive ever needs to span the monitor/trap inboxes too, that
cross-channel merge is the still-unmade decision.
- **Cooperative-only**: preemption and (future) cancellation both depend on the
actor reaching `check!()`/yield/alloc/blocking points.
- `run()` is single-thread (`Config::exact(1)`); tests rely on deterministic
single-thread ordering (parent runs until it parks). Multi-thread via
`runtime::init(Config…)`.
## Workflow expectations (from the human)
- TDD: write the failing test first, then implement.
- Commit incrementally with conventional-commit messages; keep each commit a
reviewable unit (they diff in their IDE and are the filter to the codebase).
- Run the full suite before each commit; check perf when a change touches
Slot/scheduler/spawn/finalize hot paths.