smarm/ROADMAP.md

smarm — Roadmap

v0.4 — Actor primitives ✅ DONE

1. Cooperative cancellation ✅ (a8ddb4a)

Stop flag on Actor behind Arc<AtomicBool>. Observation points: maybe_preempt/check!() + the wakeup side of park_current/yield_now. Sentinel = StopSentinel (zero-size), caught in trampoline → Outcome::Stopped. join() on a stopped actor returns Ok(()).

Known gap (by design): a tight no-alloc loop without check!() cannot be stopped, same as preemption.

2. one_for_all / rest_for_one + ordered shutdown ✅ (351dc9c)

Strategy::{OneForOne,OneForAll,RestForOne} via .strategy(). Triggering child's Restart policy decides whether anything restarts; strategy decides which siblings are cycled. Survivors stopped in reverse start order, awaited on the existing supervisor_channel funnel, restarted in start order.

Signal::Stopped(pid) + DownReason::Stopped added (distinct from Exit).

Also fixed a latent bug (e80334b): orphaned timers were gating shutdown. live == 0 is now the shutdown condition; timer heap cleared on exit.

3. Links + trap_exit ✅ (6581484)

Slot.links: Vec<Pid> (bidirectional). On finalize: abnormal death → request_stop(peer) unless peer traps, in which case deliver ExitSignal message. Normal exit does not propagate. trap_exit() returns a dedicated Receiver<ExitSignal> (distinct from monitor Down channel).

4. Selective receive ✅ (03f3875)

Receiver::recv_match(pred) + try_recv_match. Scans VecDeque front-to-back, removes+returns first match, parks and re-scans on new arrivals. pred is Fn(&T) -> bool (not FnMut; runs under the channel lock — keep it cheap).

5. Grab-bag ✅

• demonitor (5181037): monitor() → Monitor { id, target, rx }. demonitor(&m) -> Option<MonitorId>. Dropped Sender released post-lock (non-reentrant shared mutex discipline).
• gen_server (55221e9): GenServer trait, call/cast over one Envelope inbox. terminate hook via drop guard. No handle_info, no call timeout (deferred — needs cross-channel mailbox merge or per-recv deadline).
• Named registry: register/whereis/send_by_name — not yet done. HashMap in SharedState. No dependencies; can land any time.
• arm-port branch: aarch64 context-switch backend extracted into src/arch/. Untested on hardware. Merge only after on-device validation.

---

v0.5 — Runtime decomposition & run-path scaling ✅ DONE

Goal: dismantle the single Mutex<SharedState> along the run path so the runtime scales to dozens of cores, with crash isolation hardened so a torn write or a stray cancellation can never poison shared runtime state.

Guiding rules established this cycle:

• Lock count that matters is hot-path locks. Cold lifecycle paths may keep a small per-slot lock; the run path targets zero locks.
• Lock ordering: io-before-shared, slot locks are leaves (never hold two slot locks at once).
• No unwind inside a runtime critical section. Stop sentinel only fires at lock-free observation points.
• Never hold a thread-local guard across a possible switch point. An actor can be preempted at any allocation and resume on a different OS thread. with_runtime, with_shared, and every RawMutex guard disable preemption for this reason.

Phase 1 — State decomposition ✅ (3c7e26b)

next_monitor_id → AtomicU64. timers → own Mutex<Timers>. io → own Mutex<Option<IoThread>>. pending_closures → folded into Slot::pending_closure. check_cancelled() gated behind PREEMPTION_ENABLED (poison fix).

SharedState now holds only: slots, free_list, run_queue, root_pid.

Phase 2 — Slot table split ✅ (5e0c9d4)

Fixed slab Box<[Slot]>. Generation packed into state word: one AtomicU64 = (gen << 32) | state — gen check is atomic with every transition. Per-slot CAS state machine Vacant/Queued/Running/RunningNotified/Parked/Done. Per-slot RawMutex for cold collections. Free list → RawMutex<Vec<u32>>. live_actors atomic; termination = io_out==0 && queue empty && live==0.

Phase 3 — Pluggable run queue ✅ (1b3b618)

Compile-time selected via feature flags. Variants:

• rq-mutex — Mutex<VecDeque>, baseline.
• rq-mpmc — hand-rolled Vyukov bounded MPMC ring.
• rq-striped — M Vyukov rings, fetch-add ticket distribution.

All variants compile in every build; feature only picks the alias. Bounded rings are sound via slab cap + at-most-once-enqueued invariant.

Queue ops require preemption disabled (a producer suspended mid-publish stalls every consumer behind its cell — livelock).

Phase 4 — Bench harness ✅ (6d9f369)

benches/rq_micro.rs: raw structure sweep. benches/rq_runtime.rs: yield-storm, ping-pong-pairs, spawn-storm. scripts/bench_rq.sh: rebuilds per feature, aggregates to bench_results/summary.csv.

Phase 5 — Safety hardening & model checking ✅ (039703d)

State machine extracted to src/slot_state.rs for loom checking. Loom on the rings via src/sync_shim.rs. NoPreempt/no-unwind/TLS-guard audit complete. Invariant-assertion sweep: every StateWord transition self-asserts its precondition.

---

v0.6 — Actor ergonomics ✅ DONE

1. Channel mutex migration ✅ (8ff6cf4)

channel::Inner<T> migrated from std::sync::Mutex to RawMutex, closing the cross-thread pthread-release hole (guard held with preemption enabled → timeslice switch → unlock from a different OS thread, UB) and removing poison.

The strict leaf rule did not survive contact: finalize clones the supervisor/trap senders and monitor() clones the Down sender, all under a cold lock, and those senders live in the slot — the nesting is structural. The check generalized to two classes: Leaf → Channel, at most one of each, both violation directions debug-asserted at the acquisition site. Channel critical sections call only the lock-free unpark protocol, so the order is acyclic. recv_match's user predicate still runs under the channel lock — cheap/pure/channel-free, as documented.

2. Named registry ✅ (2c7cf0b)

register / whereis / unregister + the inverse name_of(pid): a bimap (two HashMaps in exact inverse, debug-asserted) under one Leaf RawMutex in RuntimeInner — the planned "HashMap in SharedState" predated the v0.5 decomposition. Erlang semantics: one name per pid, one pid per name, NameTaken only against a live holder. Cleanup is lazy: liveness rides the generation-checked slot word (pids are never reused), dead bindings behave as absent and are pruned on contact — no finalize_actor hook, no Slot field, no reset-in-three-places exposure.

send_by_name was dropped from scope deliberately: smarm has no per-pid mailbox, so there is nothing generic to send to. The registry yields a Pid for monitor/link/request_stop and for routing user-owned channels.

3. gen_server: call timeout ✅ (134ff52, 90b7040)

Landed as two primitives, without the sketched monitor machinery:

• Receiver::recv_timeout(Duration) — the per-recv deadline, on the existing timer::Reason::WaitTimeout/TimerTarget machinery exactly as Mutex::lock_timeout uses it (per-wait seq; stale heap entries no-op on seq mismatch; message-first race resolution; Disconnected distinct from Timeout).
• ServerRef::call_timeout = send + recv_timeout on the reply channel. Server death is already observable there (reply sender drops as the server unwinds), so monitor + demonitor-on-timeout had nothing left to guarantee: no registration exists, so nothing can leak. Timed-out requests are still handled; the late reply's send fails harmlessly (Erlang semantics).

(handle_info still deferred — needs cross-channel mailbox merge; see Later.)

---

Later

Tunable scheduler idle policy (RFC 004)

Today idle schedulers fall back to thread::sleep(100µs), putting ~100µs worst-case latency on every cross-thread actor handoff. RFC 004 specifies a bounded spinning-worker policy (Go/Tokio model): idle schedulers spin on a cheap atomic for a configurable budget before parking, so a burst of work is picked up in cache-coherency time rather than after a kernel reschedule.

Two new Config knobs: spin_budget_cycles (RDTSC cycles to spin; 0 recovers today's behaviour) and max_spinners (cap on concurrent spinners; default N/2). Single new atomic n_spinning. No change to the run-queue data structure — lands on the current Mutex<VecDeque> and is independent of any future queue topology work.

Full spec: rfc_004-tunable-scheduler-idle-policy.md (artefacts.kalsbeek.dev).

Unbounded / configurable-bounded actor count

Fixed slab with a loud assert (Config::max_actors(n), default 16 384). Revisit with a segmented slab (array of AtomicPtr<Segment>, doubling segment sizes, append-only) once the cap is actually hit. Do not let it calcify.

gen_server: handle_info

Needs a cross-channel mailbox merge (call/cast inbox + out-of-band signals) — the still-unmade decision between a unified per-actor mailbox and per-channel recv_match composition.

IO fd hygiene on actor death

Pre-existing v0.2 TODO in io.rs: fds registered with epoll for a dead actor are not cleaned up. Audit and fix.

arm-port validation & merge

arm-port branch carries an AAPCS64 context-switch backend, never run on hardware. Build + run full test suite on an aarch64 device; check chained_spawn / yield_many bench medians; merge and update README.

---

Invariants & gotchas (respect these across all cycles)

• Shared mutex is non-reentrant. Sender::send can call unpark → with_shared. Never send on a channel while holding the shared lock. Pattern: mem::take data under the lock, send after releasing. See finalize_actor.
• finalize_actor order: take stack/waiters/monitors under lock + set Done/outcome → recycle stack → deliver supervisor Signal + monitor Downs → unpark joiners → reclaim slot if outstanding_handles==0. Death notifications always precede reclamation.
• Slot lifecycle reset in THREE places: Slot::vacant(), reclaim_slot() (runtime.rs), slot-init block in spawn_under (scheduler.rs). Any new Slot field must be reset in all three.
• Pid = (index, generation). Stale handles caught by generation mismatch in slot()/slot_mut(). The monitor NoProc path relies on this.
• No select, no unified per-process mailbox. Why the supervisor uses the single supervisor_channel funnel. recv_match is per-channel only; a cross-channel merge is the still-unmade decision (see v0.6 #3).
• Cooperative-only. Preemption and cancellation both depend on the actor reaching check!()/yield/alloc/blocking points.
• Lock order is Leaf → Channel, one of each at most (debug-asserted in raw_mutex.rs). Leaf = cold locks / free list / stack pool / registry, mutual leaves. A channel lock may be taken under a Leaf (finalize/monitor clone senders living in slots); nothing may be locked under a channel lock.
• Queue ops require preemption disabled. A producer suspended mid-publish stalls every consumer — livelock. with_runtime, with_shared, and RawMutex guards all disable preemption for their span.
• run() is single-thread (Config::exact(1)); tests rely on deterministic single-thread ordering. Multi-thread via runtime::init(Config…).