smarm/ROADMAP.md

smarm — Roadmap

v0.4 — Actor primitives ✅ DONE

1. Cooperative cancellation ✅ (a8ddb4a)

Stop flag on Actor behind Arc<AtomicBool>. Observation points: maybe_preempt/check!() + the wakeup side of park_current/yield_now. Sentinel = StopSentinel (zero-size), caught in trampoline → Outcome::Stopped. join() on a stopped actor returns Ok(()).

Known gap (by design): a tight no-alloc loop without check!() cannot be stopped, same as preemption.

2. one_for_all / rest_for_one + ordered shutdown ✅ (351dc9c)

Strategy::{OneForOne,OneForAll,RestForOne} via .strategy(). Triggering child's Restart policy decides whether anything restarts; strategy decides which siblings are cycled. Survivors stopped in reverse start order, awaited on the existing supervisor_channel funnel, restarted in start order.

Signal::Stopped(pid) + DownReason::Stopped added (distinct from Exit).

Also fixed a latent bug (e80334b): orphaned timers were gating shutdown. live == 0 is now the shutdown condition; timer heap cleared on exit.

3. Links + trap_exit ✅ (6581484)

Slot.links: Vec<Pid> (bidirectional). On finalize: abnormal death → request_stop(peer) unless peer traps, in which case deliver ExitSignal message. Normal exit does not propagate. trap_exit() returns a dedicated Receiver<ExitSignal> (distinct from monitor Down channel).

4. Selective receive ✅ (03f3875)

Receiver::recv_match(pred) + try_recv_match. Scans VecDeque front-to-back, removes+returns first match, parks and re-scans on new arrivals. pred is Fn(&T) -> bool (not FnMut; runs under the channel lock — keep it cheap).

5. Grab-bag ✅

• demonitor (5181037): monitor() → Monitor { id, target, rx }. demonitor(&m) -> Option<MonitorId>. Dropped Sender released post-lock (non-reentrant shared mutex discipline).
• gen_server (55221e9): GenServer trait, call/cast over one Envelope inbox. terminate hook via drop guard. No handle_info, no call timeout (deferred — needs cross-channel mailbox merge or per-recv deadline).
• Named registry: register/whereis/send_by_name — not yet done. HashMap in SharedState. No dependencies; can land any time.
• arm-port branch: aarch64 context-switch backend extracted into src/arch/. Untested on hardware. Merge only after on-device validation.

---

v0.5 — Runtime decomposition & run-path scaling ✅ DONE

Goal: dismantle the single Mutex<SharedState> along the run path so the runtime scales to dozens of cores, with crash isolation hardened so a torn write or a stray cancellation can never poison shared runtime state.

Guiding rules established this cycle:

• Lock count that matters is hot-path locks. Cold lifecycle paths may keep a small per-slot lock; the run path targets zero locks.
• Lock ordering: io-before-shared, slot locks are leaves (never hold two slot locks at once).
• No unwind inside a runtime critical section. Stop sentinel only fires at lock-free observation points.
• Never hold a thread-local guard across a possible switch point. An actor can be preempted at any allocation and resume on a different OS thread. with_runtime, with_shared, and every RawMutex guard disable preemption for this reason.

Phase 1 — State decomposition ✅ (3c7e26b)

next_monitor_id → AtomicU64. timers → own Mutex<Timers>. io → own Mutex<Option<IoThread>>. pending_closures → folded into Slot::pending_closure. check_cancelled() gated behind PREEMPTION_ENABLED (poison fix).

SharedState now holds only: slots, free_list, run_queue, root_pid.

Phase 2 — Slot table split ✅ (5e0c9d4)

Fixed slab Box<[Slot]>. Generation packed into state word: one AtomicU64 = (gen << 32) | state — gen check is atomic with every transition. Per-slot CAS state machine Vacant/Queued/Running/RunningNotified/Parked/Done. Per-slot RawMutex for cold collections. Free list → RawMutex<Vec<u32>>. live_actors atomic; termination = io_out==0 && queue empty && live==0.

Phase 3 — Pluggable run queue ✅ (1b3b618)

Compile-time selected via feature flags. Variants:

• rq-mutex — Mutex<VecDeque>, baseline.
• rq-mpmc — hand-rolled Vyukov bounded MPMC ring.
• rq-striped — M Vyukov rings, fetch-add ticket distribution.

All variants compile in every build; feature only picks the alias. Bounded rings are sound via slab cap + at-most-once-enqueued invariant.

Queue ops require preemption disabled (a producer suspended mid-publish stalls every consumer behind its cell — livelock).

Phase 4 — Bench harness ✅ (6d9f369)

benches/rq_micro.rs: raw structure sweep. benches/rq_runtime.rs: yield-storm, ping-pong-pairs, spawn-storm. scripts/bench_rq.sh: rebuilds per feature, aggregates to bench_results/summary.csv.

Phase 5 — Safety hardening & model checking ✅ (039703d)

State machine extracted to src/slot_state.rs for loom checking. Loom on the rings via src/sync_shim.rs. NoPreempt/no-unwind/TLS-guard audit complete. Invariant-assertion sweep: every StateWord transition self-asserts its precondition.

---

v0.6 — Fast follows & open tracks

Priority order within the cycle; each item is its own commit.

1. Channel mutex migration (first change)

channel::Inner<T> holds a std::sync::Mutex. Phase 2 surfaced a sharp correctness issue: the MutexGuard is held with preemption enabled, so a timeslice switch inside a channel critical section migrates the actor and releases the pthread mutex from a different OS thread — UB (Linux futexes happen to tolerate it, but it's not guaranteed). The Phase-1 check_cancelled gating already removes the unwind source, so poison is safe today; this is about the cross-thread release.

Migrate to RawMutex (which disables preemption and is cross-thread-release sound by construction). recv_match runs a user predicate under this lock — keep it cheap/pure.

2. Named registry

register(name, pid) / whereis(name) -> Option<Pid> / send_by_name. HashMap<String, Pid> in SharedState, guarded by the shared lock. No architecture changes; can land immediately after #1.

3. gen_server: handle_info + call timeout

• handle_info: needs a cross-channel mailbox merge (call/cast inbox + out-of-band signals). The still-unmade decision: whether to introduce a unified per-actor mailbox or keep per-channel recv_match composition.

• Call timeout: call with a deadline — monitor the server, wait reply-or-Down-or-deadline, then demonitor+drop so a timed-out call leaks no registration. Requires a per-recv deadline / Signal::Timeout primitive that doesn't exist yet.

  These two can land independently; call timeout is smaller and more useful first.

4. Unbounded / configurable-bounded actor count

v0.5 ships a fixed slab with a loud assert (Config::max_actors(n), default 16 384). Revisit with a segmented slab (array of AtomicPtr<Segment>, doubling segment sizes, append-only) once the cap is actually hit or a workload demands it. Do not let the fixed cap calcify into an assumption.

5. Idle-wakeup eventcount

Idle scheduler threads currently poll-sleep at 100 µs. A futex-based eventcount would reduce idle latency. Defer until benches show it matters.

6. IO fd hygiene on actor death

Pre-existing v0.2 TODO in io.rs: fds registered with epoll for a dead actor are not cleaned up. Audit and fix.

7. arm-port validation & merge

arm-port branch carries an AAPCS64 context-switch backend, never run on hardware. Steps: build + run full test suite on an aarch64 device; check chained_spawn / yield_many bench medians for regressions vs x86 baseline; merge and update README module table to reflect multi-arch support.

---

Invariants & gotchas (respect these across all cycles)

• Shared mutex is non-reentrant. Sender::send can call unpark → with_shared. Never send on a channel while holding the shared lock. Pattern: mem::take data under the lock, send after releasing. See finalize_actor.
• finalize_actor order: take stack/waiters/monitors under lock + set Done/outcome → recycle stack → deliver supervisor Signal + monitor Downs → unpark joiners → reclaim slot if outstanding_handles==0. Death notifications always precede reclamation.
• Slot lifecycle reset in THREE places: Slot::vacant(), reclaim_slot() (runtime.rs), slot-init block in spawn_under (scheduler.rs). Any new Slot field must be reset in all three.
• Pid = (index, generation). Stale handles caught by generation mismatch in slot()/slot_mut(). The monitor NoProc path relies on this.
• No select, no unified per-process mailbox. Why the supervisor uses the single supervisor_channel funnel. recv_match is per-channel only; a cross-channel merge is the still-unmade decision (see v0.6 #3).
• Cooperative-only. Preemption and cancellation both depend on the actor reaching check!()/yield/alloc/blocking points.
• Queue ops require preemption disabled. A producer suspended mid-publish stalls every consumer — livelock. with_runtime, with_shared, and RawMutex guards all disable preemption for their span.
• run() is single-thread (Config::exact(1)); tests rely on deterministic single-thread ordering. Multi-thread via runtime::init(Config…).