smarm/ROADMAP.md

# smarm — Roadmap

## v0.4 — Actor primitives  ✅ DONE

### 1. Cooperative cancellation  ✅ (`a8ddb4a`)
Stop flag on `Actor` behind `Arc<AtomicBool>`. Observation points:
`maybe_preempt`/`check!()` + the wakeup side of `park_current`/`yield_now`.
Sentinel = `StopSentinel` (zero-size), caught in trampoline → `Outcome::Stopped`.
`join()` on a stopped actor returns `Ok(())`.

Known gap (by design): a tight no-alloc loop without `check!()` cannot be
stopped, same as preemption.

### 2. one_for_all / rest_for_one + ordered shutdown  ✅ (`351dc9c`)
`Strategy::{OneForOne,OneForAll,RestForOne}` via `.strategy()`. Triggering
child's `Restart` policy decides whether anything restarts; strategy decides
which siblings are cycled. Survivors stopped in reverse start order, awaited on
the existing `supervisor_channel` funnel, restarted in start order.

`Signal::Stopped(pid)` + `DownReason::Stopped` added (distinct from Exit).

Also fixed a latent bug (`e80334b`): orphaned timers were gating shutdown.
`live == 0` is now the shutdown condition; timer heap cleared on exit.

### 3. Links + trap_exit  ✅ (`6581484`)
`Slot.links: Vec<Pid>` (bidirectional). On finalize: abnormal death →
`request_stop(peer)` unless peer traps, in which case deliver `ExitSignal`
message. Normal exit does not propagate. `trap_exit()` returns a dedicated
`Receiver<ExitSignal>` (distinct from monitor `Down` channel).

### 4. Selective receive  ✅ (`03f3875`)
`Receiver::recv_match(pred)` + `try_recv_match`. Scans `VecDeque` front-to-back,
removes+returns first match, parks and re-scans on new arrivals. `pred` is
`Fn(&T) -> bool` (not `FnMut`; runs under the channel lock — keep it cheap).

### 5. Grab-bag  ✅
- **demonitor** (`5181037`): `monitor()` → `Monitor { id, target, rx }`.
  `demonitor(&m) -> Option<MonitorId>`. Dropped `Sender` released post-lock
  (non-reentrant shared mutex discipline).
- **gen_server** (`55221e9`): `GenServer` trait, `call`/`cast` over one
  `Envelope` inbox. `terminate` hook via drop guard. No `handle_info`, no call
  timeout (deferred — needs cross-channel mailbox merge or per-recv deadline).
- **Named registry**: `register`/`whereis`/`send_by_name` — **not yet done**.
  HashMap in `SharedState`. No dependencies; can land any time.
- **arm-port branch**: aarch64 context-switch backend extracted into
  `src/arch/`. Untested on hardware. Merge only after on-device validation.

---

## v0.5 — Runtime decomposition & run-path scaling  ✅ DONE

Goal: dismantle the single `Mutex<SharedState>` along the run path so the
runtime scales to dozens of cores, with crash isolation hardened so a torn
write or a stray cancellation can never poison shared runtime state.

Guiding rules established this cycle:
- **Lock count that matters is hot-path locks.** Cold lifecycle paths may keep a
  small per-slot lock; the run path targets zero locks.
- **Lock ordering: io-before-shared, slot locks are leaves** (never hold two
  slot locks at once).
- **No unwind inside a runtime critical section.** Stop sentinel only fires at
  lock-free observation points.
- **Never hold a thread-local guard across a possible switch point.** An actor
  can be preempted at any allocation and resume on a different OS thread.
  `with_runtime`, `with_shared`, and every `RawMutex` guard disable preemption
  for this reason.

### Phase 1 — State decomposition  ✅ (`3c7e26b`)
`next_monitor_id` → `AtomicU64`. `timers` → own `Mutex<Timers>`. `io` → own
`Mutex<Option<IoThread>>`. `pending_closures` → folded into `Slot::pending_closure`.
`check_cancelled()` gated behind `PREEMPTION_ENABLED` (poison fix).

`SharedState` now holds only: `slots`, `free_list`, `run_queue`, `root_pid`.

### Phase 2 — Slot table split  ✅ (`5e0c9d4`)
Fixed slab `Box<[Slot]>`. Generation packed into state word: one
`AtomicU64 = (gen << 32) | state` — gen check is atomic with every transition.
Per-slot CAS state machine `Vacant/Queued/Running/RunningNotified/Parked/Done`.
Per-slot `RawMutex` for cold collections. Free list → `RawMutex<Vec<u32>>`.
`live_actors` atomic; termination = io_out==0 && queue empty && live==0.

### Phase 3 — Pluggable run queue  ✅ (`1b3b618`)
Compile-time selected via feature flags. Variants:
- `rq-mutex` — `Mutex<VecDeque>`, baseline.
- `rq-mpmc` — hand-rolled Vyukov bounded MPMC ring.
- `rq-striped` — M Vyukov rings, fetch-add ticket distribution.

All variants compile in every build; feature only picks the alias. Bounded rings
are sound via slab cap + at-most-once-enqueued invariant.

Queue ops require preemption disabled (a producer suspended mid-publish stalls
every consumer behind its cell — livelock).

### Phase 4 — Bench harness  ✅ (`6d9f369`)
`benches/rq_micro.rs`: raw structure sweep. `benches/rq_runtime.rs`:
yield-storm, ping-pong-pairs, spawn-storm. `scripts/bench_rq.sh`: rebuilds per
feature, aggregates to `bench_results/summary.csv`.

### Phase 5 — Safety hardening & model checking  ✅ (`039703d`)
State machine extracted to `src/slot_state.rs` for loom checking. Loom on the
rings via `src/sync_shim.rs`. `NoPreempt`/no-unwind/TLS-guard audit complete.
Invariant-assertion sweep: every `StateWord` transition self-asserts its
precondition.

---

## v0.6 — Actor ergonomics  ✅ DONE

### 1. Channel mutex migration  ✅ (`8ff6cf4`)
`channel::Inner<T>` migrated from `std::sync::Mutex` to `RawMutex`, closing
the cross-thread pthread-release hole (guard held with preemption enabled →
timeslice switch → unlock from a different OS thread, UB) and removing poison.

The strict leaf rule did not survive contact: finalize clones the
supervisor/trap senders and `monitor()` clones the Down sender, all under a
cold lock, and those senders *live in the slot* — the nesting is structural.
The check generalized to two classes: **Leaf → Channel**, at most one of
each, both violation directions debug-asserted at the acquisition site.
Channel critical sections call only the lock-free unpark protocol, so the
order is acyclic. `recv_match`'s user predicate still runs under the channel
lock — cheap/pure/channel-free, as documented.

### 2. Named registry  ✅ (`2c7cf0b`)
`register` / `whereis` / `unregister` + the inverse `name_of(pid)`: a
**bimap** (two HashMaps in exact inverse, debug-asserted) under one Leaf
`RawMutex` in `RuntimeInner` — the planned "HashMap in SharedState" predated
the v0.5 decomposition. Erlang semantics: one name per pid, one pid per
name, NameTaken only against a *live* holder. Cleanup is lazy: liveness
rides the generation-checked slot word (pids are never reused), dead
bindings behave as absent and are pruned on contact — no `finalize_actor`
hook, no Slot field, no reset-in-three-places exposure.

`send_by_name` was dropped from scope deliberately: smarm has no per-pid
mailbox, so there is nothing generic to send *to*. The registry yields a
`Pid` for monitor/link/request_stop and for routing user-owned channels.

### 3. gen_server: call timeout  ✅ (`134ff52`, `90b7040`)
Landed as two primitives, *without* the sketched monitor machinery:

- `Receiver::recv_timeout(Duration)` — the per-`recv` deadline, on the
  existing `timer::Reason::WaitTimeout`/`TimerTarget` machinery exactly as
  `Mutex::lock_timeout` uses it (per-wait seq; stale heap entries no-op on
  seq mismatch; message-first race resolution; `Disconnected` distinct from
  `Timeout`).
- `ServerRef::call_timeout` = send + `recv_timeout` on the reply channel.
  Server death is already observable there (reply sender drops as the server
  unwinds), so monitor + demonitor-on-timeout had nothing left to guarantee:
  no registration exists, so nothing can leak. Timed-out requests are still
  handled; the late reply's send fails harmlessly (Erlang semantics).

(`handle_info` still deferred — needs cross-channel mailbox merge; see Later.)

---

## Later

### Tunable scheduler idle policy (RFC 004)
Today idle schedulers fall back to `thread::sleep(100µs)`, putting ~100µs
worst-case latency on every cross-thread actor handoff. RFC 004 specifies a
bounded spinning-worker policy (Go/Tokio model): idle schedulers spin on a
cheap atomic for a configurable budget before parking, so a burst of work is
picked up in cache-coherency time rather than after a kernel reschedule.

Two new `Config` knobs: `spin_budget_cycles` (RDTSC cycles to spin; `0`
recovers today's behaviour) and `max_spinners` (cap on concurrent spinners;
default `N/2`). Single new atomic `n_spinning`. No change to the run-queue
data structure — lands on the current `Mutex<VecDeque>` and is independent of
any future queue topology work.

Full spec: `rfc_004-tunable-scheduler-idle-policy.md` (artefacts.kalsbeek.dev).

### Unbounded / configurable-bounded actor count
Fixed slab with a loud assert (`Config::max_actors(n)`, default 16 384).
Revisit with a segmented slab (array of `AtomicPtr<Segment>`, doubling segment
sizes, append-only) once the cap is actually hit. Do not let it calcify.

### gen_server: handle_info
Needs a cross-channel mailbox merge (call/cast inbox + out-of-band signals) —
the still-unmade decision between a unified per-actor mailbox and per-channel
`recv_match` composition.

### IO fd hygiene on actor death
Pre-existing v0.2 TODO in `io.rs`: fds registered with epoll for a dead actor
are not cleaned up. Audit and fix.

### arm-port validation & merge
`arm-port` branch carries an AAPCS64 context-switch backend, never run on
hardware. Build + run full test suite on an aarch64 device; check
`chained_spawn` / `yield_many` bench medians; merge and update README.

---

## Invariants & gotchas (respect these across all cycles)

- **Shared mutex is non-reentrant.** `Sender::send` can call `unpark` →
  `with_shared`. Never send on a channel while holding the shared lock. Pattern:
  `mem::take` data under the lock, send after releasing. See `finalize_actor`.
- **`finalize_actor` order:** take stack/waiters/monitors under lock + set
  Done/outcome → recycle stack → deliver supervisor Signal + monitor Downs →
  unpark joiners → reclaim slot if `outstanding_handles==0`. Death notifications
  always precede reclamation.
- **Slot lifecycle reset in THREE places:** `Slot::vacant()`, `reclaim_slot()`
  (runtime.rs), slot-init block in `spawn_under` (scheduler.rs). Any new `Slot`
  field must be reset in all three.
- **Pid = (index, generation).** Stale handles caught by generation mismatch in
  `slot()/slot_mut()`. The monitor `NoProc` path relies on this.
- **No `select`, no unified per-process mailbox.** Why the supervisor uses the
  single `supervisor_channel` funnel. `recv_match` is per-channel only; a
  cross-channel merge is the still-unmade decision (see v0.6 #3).
- **Cooperative-only.** Preemption and cancellation both depend on the actor
  reaching `check!()`/yield/alloc/blocking points.
- **Lock order is Leaf → Channel, one of each at most** (debug-asserted in
  `raw_mutex.rs`). Leaf = cold locks / free list / stack pool / registry,
  mutual leaves. A channel lock may be taken under a Leaf (finalize/monitor
  clone senders living in slots); nothing may be locked under a channel lock.
- **Queue ops require preemption disabled.** A producer suspended mid-publish
  stalls every consumer — livelock. `with_runtime`, `with_shared`, and
  `RawMutex` guards all disable preemption for their span.
- **`run()` is single-thread** (`Config::exact(1)`); tests rely on deterministic
  single-thread ordering. Multi-thread via `runtime::init(Config…)`.