plan: roadmap refresh

ROADMAP.md

@@ -34,23 +34,15 @@ Commits `e5d1b3b`, `24b95c9`, `f6969e5`.
 The run-queue shootout (harness `6d9f369`, 24-core sweep 1–24 schedulers,
 report `bench_report_rq_shootout.html`) landed in **RFC 005's World 3**: the
 three queue variants are within 10–15% of each other in `rq_runtime` at every
-scheduler count ≥ 4, on all three workloads. Striped's real micro-bench
+scheduler count ≥ 4, on all three workloads. 
-dominance (24.8 M ops/s at 24t, 4.3× over mutex) does not propagate to the
-runtime because the queue is not the hot path in any measured workload. USL
-fits put σ at 5.5–7.6 (super-serial) for ping-pong: the ceiling is per-wake
-latency through the park/unpark protocol, not queue push/pop.
-
 Consequences:
 - **`rq-mutex` stays the default** — simplest correct, no capacity
-  constraints, locking model already integrated with the preemption-disabled
+  constraints, locking model already integrated.
-  queue-op invariant.
 - **Feature plumbing stays as is.** All three variants keep compiling in
   every build; `rq-mpmc`/`rq-striped` remain selectable for benching.
 - **Reopening is benchmark-driven only.** The report documents the
   conditional upgrade paths if a future workload qualifies: mpmc for
-  message-passing-dominant loads at N ≤ 8 (1.3× lower σ, 0.16 µs ping-pong
+  message-passing-dominant loads at N ≤ 8; striped for high-contention balanced push/pop at N ≥ 16. Neither is a scheduler workload as measured.
-  at 1t); striped for high-contention balanced push/pop at N ≥ 16 — neither
-  is a scheduler workload as measured.
 - **Effort redirects to the wake path**: RFC 005 (billed as a latency patch,
   per its own World 3 framing), RFC 004, and eventually per-switch cost.
 
@@ -94,6 +86,7 @@ it's a Config knob, not a feature rebuild). Same sweep as the rq shootout.
   is pop-path overhead.
 - **spawn-storm** — neutrality check; spawns bypass the slot by policy.
 Acceptance flips the default on and re-baselines for RFC 004.
+This is done, see results annotated in RFC005.
 
 ### 3. Spinning workers (RFC 004)
 Bounded spin-before-park for idle schedulers, killing the ~100 µs
@@ -101,6 +94,7 @@ Bounded spin-before-park for idle schedulers, killing the ~100 µs
 `spin_budget_cycles` (0 recovers today's behaviour) and `max_spinners`
 (default N/2); one new atomic `n_spinning`. No run-queue changes — lands on
 the frozen `rq-mutex` substrate, independent of the slot mechanically.
+Credit for the idea: Dennis Gustafsson - [Parallelizing the physics solver BSC 2025](https://www.youtube.com/watch?v=Kvsvd67XUKw). 
 
 ### 4. RFC 004 bench + interaction pass
 Re-run the sweep with spinning enabled, slot on and off. The known
@@ -112,8 +106,52 @@ shared instead" check. In scope only if the data demands it.
 ---
 
 ## Later
+### Highest priority
+#### Process groups: the primitive pubsub & channels should have sat on
+Context: urus is a webserver written on top of smarm to provide a testing target.
+A named pid→multiset map with monitor-backed removal: `registry.rs` generalised
+from name↔pid *bimap* to name→*multiset*, the death hook reused verbatim. Local
+first. The point is that urus pubsub collapses into a pg consumer (`subscribe` =
+join, `broadcast` = send-to-members) instead of being a bespoke mechanism, and the
+same group set reads two ways — fan-out (all members) vs discovery/pool (one
+member), with different netsplit consequences. Urus shipped pubsub/channels predate this
+and want reframing on top of it. Foundational, so early in the post-v0.9 stack.
+Needs an RFC.
 
-### Unwakeable idle sleep when io is absent (terminal-wake residual)
+#### Per-switch cost (context shims, epoch protocol)
+The shootout's residual: per-wake latency is 0.16–0.18 µs at N=1 and
+0.8–1.2 µs at N=8+, dominated by the context-switch shims and the epoch
+protocol, not the queue. On current evidence this is the larger constant —
+"the whole game" alongside the v0.9 work — but there is no spec yet. Needs a
+profiling spike (where do the cycles actually go per park/unpark round-trip)
+and then an RFC before it can be scheduled.
+
+#### send_after / cancel_timer
+Message-delivery timer on the existing min-heap (`timer.rs`): deliver a value to a
+channel at a deadline, cancellable. Unlocks the gen_server idioms with no clean
+expression today — heartbeat, debounce, retry backoff, session expiry. Small; 
+
+#### Introspection — process_info / get_state / tree dump
+`trace.rs` is the seed. What an actor is parked on, queue depth, stack size; a
+gen_server state snapshot; a supervision-tree walk. The native edge over tokio —
+actors already carry pid, name, parent where tokio tasks are anonymous — so it
+costs little and differentiates a lot. Needs an RFC.
+
+#### Worker pool behaviour
+Supervised, interchangeable workers with restart semantics over a shared inbox
+(poolboy / NimblePool shape) — distinct from connection pools (bb8/deadpool), which
+pool *resources*, not *supervised processes*. Sits on `supervisor.rs` +
+`gen_server.rs`. Needs an RFC.
+
+### Medium Priority
+
+#### Demand-driven pipelines — GenStage / Broadway shape
+Supervised producer/consumer stages where consumers signal demand upstream, with
+batching, ack, partitioning. The clearest thing hex has and crates.io lacks (stream
+combinators and bounded channels are not a supervised demand-contract stage graph),
+and the natural fit for ingestion-shaped workloads. Builds on channels + gen_server
++ supervisor. Needs an RFC.
+#### Unwakeable idle sleep when io is absent (terminal-wake residual)
 The `(Some(deadline), None)` idle branch — timers pending, io subsystem never
 initialized — blocks in `thread::sleep` with no wake mechanism at all. The
 terminal wake (writes the wake pipe at AllDone) cannot reach it: no io, no
@@ -124,33 +162,49 @@ turns idle into periodic wakeups), or (b) park the branch on a condvar/futex
 the AllDone path signals — and at that point consider making the condvar the
 idle primitive for the no-io runtime generally (a cross-thread unpark could
 signal it too, see below). Decide before any no-io deployment.
-
+#### Cross-thread unpark
-### Cross-thread unpark
 `RuntimeInner::enqueue` does not wake idle sibling schedulers — only io
 completions write the wake pipe. Mid-flight this is masked (the enqueuing
 thread is awake and eats the work itself), but it costs parallelism: work
-enqueued by a busy thread waits until the sibling's idle poll times out.
+enqueued by a busy thread waits until the sibling's idle poll times out. Needs bench evidence (does the shared-queue handoff latency actually show up?) before a mechanism is picked.
-Candidate from the urus chunk-2 session; needs bench evidence (does the
+#### Unbounded / configurable-bounded actor count
-shared-queue handoff latency actually show up?) before a mechanism is picked.
-
-### Per-switch cost (context shims, epoch protocol)
-The shootout's residual: per-wake latency is 0.16–0.18 µs at N=1 and
-0.8–1.2 µs at N=8+, dominated by the context-switch shims and the epoch
-protocol, not the queue. On current evidence this is the larger constant —
-"the whole game" alongside the v0.9 work — but there is no spec yet. Needs a
-profiling spike (where do the cycles actually go per park/unpark round-trip)
-and then an RFC before it can be scheduled.
-
-### Unbounded / configurable-bounded actor count
 Fixed slab with a loud assert (`Config::max_actors(n)`, default 16 384).
 Revisit with a segmented slab (array of `AtomicPtr<Segment>`, doubling segment
 sizes, append-only) once the cap is actually hit. Do not let it calcify.
-
+#### arm-port validation & merge
-### arm-port validation & merge
 `arm-port` branch carries an AAPCS64 context-switch backend, never run on
 hardware. Build + run full test suite on an aarch64 device; check
 `chained_spawn` / `yield_many` bench medians; merge and update README.
 
+### Low priority
+#### gen_statem — postponement + state timeouts only
+A thin layer over gen_server, not a new behaviour. The (state, event) dispatch
+matrix is free from the type system and not worth porting. The two mechanisms that
+are: event **postponement** (defer events in the wrong state, replay on transition
+— selective receive, codified) and **state timeouts** (auto-cancel on state
+change). Device-connection FSMs are the canonical use. Wants send_after underneath.
+Needs an RFC. 
+
+#### Clustering — distribution epic
+Sequenced deliberately after v0.9 and the per-switch-cost spike. A fat stack of
+RFCs, not one. Spine settled in discussion; decisions still open:
+- **Explicit remote boundary, never transparency.** Serialization colours *edges*
+  (channel types), not functions — local edges stay zero-copy `Send`, only remote
+  edges take a `RemoteRef<T: Serialize + DeserializeOwned>`. No hidden latency when
+  a peer migrates; the refactor is visible by construction.
+- **One binary, role as runtime config** (`ROLE=… REGION=… SEEDS=…`); a build-hash
+  handshake enforces same-binary type identity and sidesteps cross-version type
+  agreement. Roles select which supervision subtree mounts.
+- **Distributed pg falls out of local pg + a membership/gossip layer**, and
+  distributed pubsub falls out of that for free; per-member metadata (region, load)
+  enables fly-style nearest-member routing.
+- **Migratable gen_servers** as a sub-layer: only behaviours migrate (a raw actor's
+  stack is opaque; a gen_server *between callbacks* is just its `State`), gated by
+  `Serialize` bounds + an `on_arrive` reacquire hook, addressed by name not pid. The
+  BEAM can't do this — leaning on the behaviour layer is what buys it. Requires `State` to be serializable, so should probaby spec a `trait MigratableGenServer: GenServer where Self::State: Migratable` , or something to that extent, so we can lean on the type system to make sure we don't accidentally make state that cannot be serialised. 
+- **CRDT presence** is the high-value, genuinely-hard layer above distributed pg,
+  kept *out* of the pg primitive (the pg2 strong-consistency lesson). Furthest out. Can maybe defer to rust ecosystem
+
 
 --- 
 
@@ -202,3 +256,4 @@ Agent working on urus (see same git server as smarm) reported a lazily spawned a
   `RawMutex` guards all disable preemption for their span.
 - **`run()` is single-thread** (`Config::exact(1)`); tests rely on deterministic
   single-thread ordering. Multi-thread via `runtime::init(Config…)`.
+