smarm/ROADMAP.md

# smarm — Roadmap

## v0.4 — Actor primitives  ✅ DONE

### 1. Cooperative cancellation  ✅ (`a8ddb4a`)
Stop flag on `Actor` behind `Arc<AtomicBool>`. Observation points:
`maybe_preempt`/`check!()` + the wakeup side of `park_current`/`yield_now`.
Sentinel = `StopSentinel` (zero-size), caught in trampoline → `Outcome::Stopped`.
`join()` on a stopped actor returns `Ok(())`.

Known gap (by design): a tight no-alloc loop without `check!()` cannot be
stopped, same as preemption.

### 2. one_for_all / rest_for_one + ordered shutdown  ✅ (`351dc9c`)
`Strategy::{OneForOne,OneForAll,RestForOne}` via `.strategy()`. Triggering
child's `Restart` policy decides whether anything restarts; strategy decides
which siblings are cycled. Survivors stopped in reverse start order, awaited on
the existing `supervisor_channel` funnel, restarted in start order.

`Signal::Stopped(pid)` + `DownReason::Stopped` added (distinct from Exit).

Also fixed a latent bug (`e80334b`): orphaned timers were gating shutdown.
`live == 0` is now the shutdown condition; timer heap cleared on exit.

### 3. Links + trap_exit  ✅ (`6581484`)
`Slot.links: Vec<Pid>` (bidirectional). On finalize: abnormal death →
`request_stop(peer)` unless peer traps, in which case deliver `ExitSignal`
message. Normal exit does not propagate. `trap_exit()` returns a dedicated
`Receiver<ExitSignal>` (distinct from monitor `Down` channel).

### 4. Selective receive  ✅ (`03f3875`)
`Receiver::recv_match(pred)` + `try_recv_match`. Scans `VecDeque` front-to-back,
removes+returns first match, parks and re-scans on new arrivals. `pred` is
`Fn(&T) -> bool` (not `FnMut`; runs under the channel lock — keep it cheap).

### 5. Grab-bag  ✅
- **demonitor** (`5181037`): `monitor()` → `Monitor { id, target, rx }`.
  `demonitor(&m) -> Option<MonitorId>`. Dropped `Sender` released post-lock
  (non-reentrant shared mutex discipline).
- **gen_server** (`55221e9`): `GenServer` trait, `call`/`cast` over one
  `Envelope` inbox. `terminate` hook via drop guard. No `handle_info`, no call
  timeout (deferred — needs cross-channel mailbox merge or per-recv deadline).
- **Named registry**: `register`/`whereis`/`send_by_name` — **not yet done**.
  HashMap in `SharedState`. No dependencies; can land any time.
- **arm-port branch**: aarch64 context-switch backend extracted into
  `src/arch/`. Untested on hardware. Merge only after on-device validation.

---

## v0.5 — Runtime decomposition & run-path scaling  ✅ DONE

Goal: dismantle the single `Mutex<SharedState>` along the run path so the
runtime scales to dozens of cores, with crash isolation hardened so a torn
write or a stray cancellation can never poison shared runtime state.

Guiding rules established this cycle:
- **Lock count that matters is hot-path locks.** Cold lifecycle paths may keep a
  small per-slot lock; the run path targets zero locks.
- **Lock ordering: io-before-shared, slot locks are leaves** (never hold two
  slot locks at once).
- **No unwind inside a runtime critical section.** Stop sentinel only fires at
  lock-free observation points.
- **Never hold a thread-local guard across a possible switch point.** An actor
  can be preempted at any allocation and resume on a different OS thread.
  `with_runtime`, `with_shared`, and every `RawMutex` guard disable preemption
  for this reason.

### Phase 1 — State decomposition  ✅ (`3c7e26b`)
`next_monitor_id` → `AtomicU64`. `timers` → own `Mutex<Timers>`. `io` → own
`Mutex<Option<IoThread>>`. `pending_closures` → folded into `Slot::pending_closure`.
`check_cancelled()` gated behind `PREEMPTION_ENABLED` (poison fix).

`SharedState` now holds only: `slots`, `free_list`, `run_queue`, `root_pid`.

### Phase 2 — Slot table split  ✅ (`5e0c9d4`)
Fixed slab `Box<[Slot]>`. Generation packed into state word: one
`AtomicU64 = (gen << 32) | state` — gen check is atomic with every transition.
Per-slot CAS state machine `Vacant/Queued/Running/RunningNotified/Parked/Done`.
Per-slot `RawMutex` for cold collections. Free list → `RawMutex<Vec<u32>>`.
`live_actors` atomic; termination = io_out==0 && queue empty && live==0.

### Phase 3 — Pluggable run queue  ✅ (`1b3b618`)
Compile-time selected via feature flags. Variants:
- `rq-mutex` — `Mutex<VecDeque>`, baseline.
- `rq-mpmc` — hand-rolled Vyukov bounded MPMC ring.
- `rq-striped` — M Vyukov rings, fetch-add ticket distribution.

All variants compile in every build; feature only picks the alias. Bounded rings
are sound via slab cap + at-most-once-enqueued invariant.

Queue ops require preemption disabled (a producer suspended mid-publish stalls
every consumer behind its cell — livelock).

### Phase 4 — Bench harness  ✅ (`6d9f369`)
`benches/rq_micro.rs`: raw structure sweep. `benches/rq_runtime.rs`:
yield-storm, ping-pong-pairs, spawn-storm. `scripts/bench_rq.sh`: rebuilds per
feature, aggregates to `bench_results/summary.csv`.

### Phase 5 — Safety hardening & model checking  ✅ (`039703d`)
State machine extracted to `src/slot_state.rs` for loom checking. Loom on the
rings via `src/sync_shim.rs`. `NoPreempt`/no-unwind/TLS-guard audit complete.
Invariant-assertion sweep: every `StateWord` transition self-asserts its
precondition.

---

## v0.6 — Up next

### 1. Channel mutex migration  (first change)
`channel::Inner<T>` holds a `std::sync::Mutex`. Phase 2 surfaced a sharp
correctness issue: the `MutexGuard` is held with preemption **enabled**, so a
timeslice switch inside a channel critical section migrates the actor and
releases the pthread mutex from a different OS thread — UB (Linux futexes happen
to tolerate it, but it's not guaranteed). The Phase-1 `check_cancelled` gating
already removes the unwind source, so poison is safe today; this is about the
cross-thread release.

Migrate to `RawMutex` (which disables preemption and is cross-thread-release
sound by construction). `recv_match` runs a user predicate under this lock —
keep it cheap/pure.

### 2. Named registry
`register(name, pid)` / `whereis(name) -> Option<Pid>` / `send_by_name`.
`HashMap<String, Pid>` in `SharedState`, guarded by the shared lock. No
architecture changes; can land immediately after #1.

### 3. gen_server: call timeout
`call` with a deadline — monitor the server, wait reply-or-Down-or-deadline,
then `demonitor`+drop so a timed-out call leaks no registration. Requires a
per-`recv` deadline / `Signal::Timeout` primitive that doesn't exist yet.
(`handle_info` deferred — needs cross-channel mailbox merge; see Later.)

---

## Later

### Tunable scheduler idle policy (RFC 004)
Today idle schedulers fall back to `thread::sleep(100µs)`, putting ~100µs
worst-case latency on every cross-thread actor handoff. RFC 004 specifies a
bounded spinning-worker policy (Go/Tokio model): idle schedulers spin on a
cheap atomic for a configurable budget before parking, so a burst of work is
picked up in cache-coherency time rather than after a kernel reschedule.

Two new `Config` knobs: `spin_budget_cycles` (RDTSC cycles to spin; `0`
recovers today's behaviour) and `max_spinners` (cap on concurrent spinners;
default `N/2`). Single new atomic `n_spinning`. No change to the run-queue
data structure — lands on the current `Mutex<VecDeque>` and is independent of
any future queue topology work.

Full spec: `rfc_004-tunable-scheduler-idle-policy.md` (artefacts.kalsbeek.dev).

### Unbounded / configurable-bounded actor count
Fixed slab with a loud assert (`Config::max_actors(n)`, default 16 384).
Revisit with a segmented slab (array of `AtomicPtr<Segment>`, doubling segment
sizes, append-only) once the cap is actually hit. Do not let it calcify.

### gen_server: handle_info
Needs a cross-channel mailbox merge (call/cast inbox + out-of-band signals) —
the still-unmade decision between a unified per-actor mailbox and per-channel
`recv_match` composition.

### IO fd hygiene on actor death
Pre-existing v0.2 TODO in `io.rs`: fds registered with epoll for a dead actor
are not cleaned up. Audit and fix.

### arm-port validation & merge
`arm-port` branch carries an AAPCS64 context-switch backend, never run on
hardware. Build + run full test suite on an aarch64 device; check
`chained_spawn` / `yield_many` bench medians; merge and update README.

---

## Invariants & gotchas (respect these across all cycles)

- **Shared mutex is non-reentrant.** `Sender::send` can call `unpark` →
  `with_shared`. Never send on a channel while holding the shared lock. Pattern:
  `mem::take` data under the lock, send after releasing. See `finalize_actor`.
- **`finalize_actor` order:** take stack/waiters/monitors under lock + set
  Done/outcome → recycle stack → deliver supervisor Signal + monitor Downs →
  unpark joiners → reclaim slot if `outstanding_handles==0`. Death notifications
  always precede reclamation.
- **Slot lifecycle reset in THREE places:** `Slot::vacant()`, `reclaim_slot()`
  (runtime.rs), slot-init block in `spawn_under` (scheduler.rs). Any new `Slot`
  field must be reset in all three.
- **Pid = (index, generation).** Stale handles caught by generation mismatch in
  `slot()/slot_mut()`. The monitor `NoProc` path relies on this.
- **No `select`, no unified per-process mailbox.** Why the supervisor uses the
  single `supervisor_channel` funnel. `recv_match` is per-channel only; a
  cross-channel merge is the still-unmade decision (see v0.6 #3).
- **Cooperative-only.** Preemption and cancellation both depend on the actor
  reaching `check!()`/yield/alloc/blocking points.
- **Queue ops require preemption disabled.** A producer suspended mid-publish
  stalls every consumer — livelock. `with_runtime`, `with_shared`, and
  `RawMutex` guards all disable preemption for their span.
- **`run()` is single-thread** (`Config::exact(1)`); tests rely on deterministic
  single-thread ordering. Multi-thread via `runtime::init(Config…)`.