diff --git a/ROADMAP.md b/ROADMAP.md index 340698a..f5be0b5 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -34,23 +34,15 @@ Commits `e5d1b3b`, `24b95c9`, `f6969e5`. The run-queue shootout (harness `6d9f369`, 24-core sweep 1–24 schedulers, report `bench_report_rq_shootout.html`) landed in **RFC 005's World 3**: the three queue variants are within 10–15% of each other in `rq_runtime` at every -scheduler count ≥ 4, on all three workloads. Striped's real micro-bench -dominance (24.8 M ops/s at 24t, 4.3× over mutex) does not propagate to the -runtime because the queue is not the hot path in any measured workload. USL -fits put σ at 5.5–7.6 (super-serial) for ping-pong: the ceiling is per-wake -latency through the park/unpark protocol, not queue push/pop. - +scheduler count ≥ 4, on all three workloads. Consequences: - **`rq-mutex` stays the default** — simplest correct, no capacity - constraints, locking model already integrated with the preemption-disabled - queue-op invariant. + constraints, locking model already integrated. - **Feature plumbing stays as is.** All three variants keep compiling in every build; `rq-mpmc`/`rq-striped` remain selectable for benching. - **Reopening is benchmark-driven only.** The report documents the conditional upgrade paths if a future workload qualifies: mpmc for - message-passing-dominant loads at N ≤ 8 (1.3× lower σ, 0.16 µs ping-pong - at 1t); striped for high-contention balanced push/pop at N ≥ 16 — neither - is a scheduler workload as measured. + message-passing-dominant loads at N ≤ 8; striped for high-contention balanced push/pop at N ≥ 16. Neither is a scheduler workload as measured. - **Effort redirects to the wake path**: RFC 005 (billed as a latency patch, per its own World 3 framing), RFC 004, and eventually per-switch cost. @@ -94,6 +86,7 @@ it's a Config knob, not a feature rebuild). Same sweep as the rq shootout. is pop-path overhead. - **spawn-storm** — neutrality check; spawns bypass the slot by policy. Acceptance flips the default on and re-baselines for RFC 004. +This is done, see results annotated in RFC005. ### 3. Spinning workers (RFC 004) Bounded spin-before-park for idle schedulers, killing the ~100 µs @@ -101,6 +94,7 @@ Bounded spin-before-park for idle schedulers, killing the ~100 µs `spin_budget_cycles` (0 recovers today's behaviour) and `max_spinners` (default N/2); one new atomic `n_spinning`. No run-queue changes — lands on the frozen `rq-mutex` substrate, independent of the slot mechanically. +Credit for the idea: Dennis Gustafsson - [Parallelizing the physics solver BSC 2025](https://www.youtube.com/watch?v=Kvsvd67XUKw). ### 4. RFC 004 bench + interaction pass Re-run the sweep with spinning enabled, slot on and off. The known @@ -112,8 +106,52 @@ shared instead" check. In scope only if the data demands it. --- ## Later +### Highest priority +#### Process groups: the primitive pubsub & channels should have sat on +Context: urus is a webserver written on top of smarm to provide a testing target. +A named pid→multiset map with monitor-backed removal: `registry.rs` generalised +from name↔pid *bimap* to name→*multiset*, the death hook reused verbatim. Local +first. The point is that urus pubsub collapses into a pg consumer (`subscribe` = +join, `broadcast` = send-to-members) instead of being a bespoke mechanism, and the +same group set reads two ways — fan-out (all members) vs discovery/pool (one +member), with different netsplit consequences. Urus shipped pubsub/channels predate this +and want reframing on top of it. Foundational, so early in the post-v0.9 stack. +Needs an RFC. -### Unwakeable idle sleep when io is absent (terminal-wake residual) +#### Per-switch cost (context shims, epoch protocol) +The shootout's residual: per-wake latency is 0.16–0.18 µs at N=1 and +0.8–1.2 µs at N=8+, dominated by the context-switch shims and the epoch +protocol, not the queue. On current evidence this is the larger constant — +"the whole game" alongside the v0.9 work — but there is no spec yet. Needs a +profiling spike (where do the cycles actually go per park/unpark round-trip) +and then an RFC before it can be scheduled. + +#### send_after / cancel_timer +Message-delivery timer on the existing min-heap (`timer.rs`): deliver a value to a +channel at a deadline, cancellable. Unlocks the gen_server idioms with no clean +expression today — heartbeat, debounce, retry backoff, session expiry. Small; + +#### Introspection — process_info / get_state / tree dump +`trace.rs` is the seed. What an actor is parked on, queue depth, stack size; a +gen_server state snapshot; a supervision-tree walk. The native edge over tokio — +actors already carry pid, name, parent where tokio tasks are anonymous — so it +costs little and differentiates a lot. Needs an RFC. + +#### Worker pool behaviour +Supervised, interchangeable workers with restart semantics over a shared inbox +(poolboy / NimblePool shape) — distinct from connection pools (bb8/deadpool), which +pool *resources*, not *supervised processes*. Sits on `supervisor.rs` + +`gen_server.rs`. Needs an RFC. + +### Medium Priority + +#### Demand-driven pipelines — GenStage / Broadway shape +Supervised producer/consumer stages where consumers signal demand upstream, with +batching, ack, partitioning. The clearest thing hex has and crates.io lacks (stream +combinators and bounded channels are not a supervised demand-contract stage graph), +and the natural fit for ingestion-shaped workloads. Builds on channels + gen_server ++ supervisor. Needs an RFC. +#### Unwakeable idle sleep when io is absent (terminal-wake residual) The `(Some(deadline), None)` idle branch — timers pending, io subsystem never initialized — blocks in `thread::sleep` with no wake mechanism at all. The terminal wake (writes the wake pipe at AllDone) cannot reach it: no io, no @@ -124,33 +162,49 @@ turns idle into periodic wakeups), or (b) park the branch on a condvar/futex the AllDone path signals — and at that point consider making the condvar the idle primitive for the no-io runtime generally (a cross-thread unpark could signal it too, see below). Decide before any no-io deployment. - -### Cross-thread unpark +#### Cross-thread unpark `RuntimeInner::enqueue` does not wake idle sibling schedulers — only io completions write the wake pipe. Mid-flight this is masked (the enqueuing thread is awake and eats the work itself), but it costs parallelism: work -enqueued by a busy thread waits until the sibling's idle poll times out. -Candidate from the urus chunk-2 session; needs bench evidence (does the -shared-queue handoff latency actually show up?) before a mechanism is picked. - -### Per-switch cost (context shims, epoch protocol) -The shootout's residual: per-wake latency is 0.16–0.18 µs at N=1 and -0.8–1.2 µs at N=8+, dominated by the context-switch shims and the epoch -protocol, not the queue. On current evidence this is the larger constant — -"the whole game" alongside the v0.9 work — but there is no spec yet. Needs a -profiling spike (where do the cycles actually go per park/unpark round-trip) -and then an RFC before it can be scheduled. - -### Unbounded / configurable-bounded actor count +enqueued by a busy thread waits until the sibling's idle poll times out. Needs bench evidence (does the shared-queue handoff latency actually show up?) before a mechanism is picked. +#### Unbounded / configurable-bounded actor count Fixed slab with a loud assert (`Config::max_actors(n)`, default 16 384). Revisit with a segmented slab (array of `AtomicPtr`, doubling segment sizes, append-only) once the cap is actually hit. Do not let it calcify. - -### arm-port validation & merge +#### arm-port validation & merge `arm-port` branch carries an AAPCS64 context-switch backend, never run on hardware. Build + run full test suite on an aarch64 device; check `chained_spawn` / `yield_many` bench medians; merge and update README. +### Low priority +#### gen_statem — postponement + state timeouts only +A thin layer over gen_server, not a new behaviour. The (state, event) dispatch +matrix is free from the type system and not worth porting. The two mechanisms that +are: event **postponement** (defer events in the wrong state, replay on transition +— selective receive, codified) and **state timeouts** (auto-cancel on state +change). Device-connection FSMs are the canonical use. Wants send_after underneath. +Needs an RFC. + +#### Clustering — distribution epic +Sequenced deliberately after v0.9 and the per-switch-cost spike. A fat stack of +RFCs, not one. Spine settled in discussion; decisions still open: +- **Explicit remote boundary, never transparency.** Serialization colours *edges* + (channel types), not functions — local edges stay zero-copy `Send`, only remote + edges take a `RemoteRef`. No hidden latency when + a peer migrates; the refactor is visible by construction. +- **One binary, role as runtime config** (`ROLE=… REGION=… SEEDS=…`); a build-hash + handshake enforces same-binary type identity and sidesteps cross-version type + agreement. Roles select which supervision subtree mounts. +- **Distributed pg falls out of local pg + a membership/gossip layer**, and + distributed pubsub falls out of that for free; per-member metadata (region, load) + enables fly-style nearest-member routing. +- **Migratable gen_servers** as a sub-layer: only behaviours migrate (a raw actor's + stack is opaque; a gen_server *between callbacks* is just its `State`), gated by + `Serialize` bounds + an `on_arrive` reacquire hook, addressed by name not pid. The + BEAM can't do this — leaning on the behaviour layer is what buys it. Requires `State` to be serializable, so should probaby spec a `trait MigratableGenServer: GenServer where Self::State: Migratable` , or something to that extent, so we can lean on the type system to make sure we don't accidentally make state that cannot be serialised. +- **CRDT presence** is the high-value, genuinely-hard layer above distributed pg, + kept *out* of the pg primitive (the pg2 strong-consistency lesson). Furthest out. Can maybe defer to rust ecosystem + --- @@ -202,3 +256,4 @@ Agent working on urus (see same git server as smarm) reported a lazily spawned a `RawMutex` guards all disable preemption for their span. - **`run()` is single-thread** (`Config::exact(1)`); tests rely on deterministic single-thread ordering. Multi-thread via `runtime::init(Config…)`. +