From 3da553b9b22c7ec9ecde06cd178f3bc828c27ae6 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 11 Jun 2026 10:48:50 +0000 Subject: [PATCH] docs(roadmap): state assessment vs smarm v0.8 + v0.2-v0.6 plan MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit urus builds clean and passes 24/24 against smarm master; the refresh is OTP adoption, not breakage. v0.2 chunks: supervised listener pool, graceful shutdown via request_stop, enforce the (currently decorative) timeouts, registry naming. Then: streaming bodies + SSE (v0.3), WebSocket upgrade (v0.4), PubSub (v0.5), Channels (v0.6). HTTP/2 stays iceboxed per spec §2.2. Two decision forks flagged inline: timeout mechanism (reaper vs smarm timed fd waits) and ws duplex topology (dup'd fds vs fd-arms-in-select) — both hinge on smarm RFC 008. --- ROADMAP.md | 190 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 190 insertions(+) create mode 100644 ROADMAP.md diff --git a/ROADMAP.md b/ROADMAP.md new file mode 100644 index 0000000..65303b5 --- /dev/null +++ b/ROADMAP.md @@ -0,0 +1,190 @@ +# urus — Roadmap + +> Companion to `urus-spec.md` (artefact store, URUS project). v1 shipped +> 2026-05-26, built against pre-v0.4 smarm — before cancellation, supervisors, +> links, gen_server, registry, select, and the slot-slab runtime existed. +> smarm is now at v0.8. This roadmap covers catching up, then extending. + +## State assessment (2026-06-11) + +Verified against smarm master (`v0.8`, slot slab + park-epoch + select): + +- **Builds clean, 24/24 tests pass.** The scheduler surface urus consumes + (`spawn`, `wait_readable`, `wait_writable`, `sleep`, `init`, `Config`, + `run`, `JoinHandle`) survived the v0.5–v0.8 runtime rework unchanged. +- **Timeouts are decorative.** `Config::{request_timeout, keep_alive_timeout}` + are threaded into `ConnLimits` and never read. `read_some` parks in + `wait_readable` with no deadline → an idle keep-alive connection pins its + actor (and its 64 KiB+ of buffers) forever. Slowloris-shaped. +- **No supervision.** Spec §2.1 calls for a supervised listener pool; + v1 predates `Strategy`/`ChildSpec` and uses bare `spawn` + `join`. + A panicking listener silently shrinks the accept pool. +- **No graceful shutdown.** Spec deferred it; smarm now has the pieces: + `request_stop` + sentinel unwind, and the 06-10 io fix (*deregister a + stopped actor's fd wait on the unwind path*) makes stopping an actor + parked in `wait_readable` safe. The `OwnedFd`-drops-on-unwind discipline + in `conn_actor.rs` already composes with this. +- **No streaming response path.** `RespBody` is `Empty | Bytes`. Blocks SSE + (a spec v1 goal that slipped) and chunked responses. +- **Full-duplex constraint (matters for WebSocket).** smarm's epoll model is + one waiter per RawFd (`AlreadyExists` on the second registration) and one + direction per parked actor. Concurrent read+write on one connection needs + either the dup-fd trick (serve.rs already uses it for the listener pool) + or a new smarm primitive. Decide in v0.2 design, before the frame codec. + +--- + +## v0.2 — OTP refresh ⏳ + +Adopt the primitives urus predates. No protocol surface changes; `Plug`/`Conn` +untouched. Chunks are independently landable, in order: + +### 1. Supervised listener pool +Replace bare spawns in `serve_with` with a supervisor: +`Strategy::OneForOne`, one `ChildSpec` per dup'd listener fd, +`Restart::Permanent`. A listener panic restarts that listener; the pool +never silently shrinks. Connection actors stay unsupervised bare spawns +(per spec: a connection is cheap and its failure is local — a 500 path, +not a restart path). + +Open question: should the restarted child re-dup from listener 0's fd, or +own its fd across restarts via the ChildSpec closure? Leaning: closure +captures the `OwnedFd`, re-registers on restart. No re-dup needed. + +### 2. Graceful shutdown +`serve_with` gains a shutdown signal (an `urus::Handle` with `.shutdown()`, +backed by a channel + `request_stop` fan-out): +1. Stop listeners (no new accepts; listener fds close). +2. Stop connection actors — either drain-then-stop with a deadline, or + immediate `request_stop` (unwind closes fds cleanly via `OwnedFd::drop`). +3. Supervisor winds down; `rt.run` returns; `serve_with` returns. + +Requires tracking live connection pids. Cheapest honest version: a +`gen_server` connection registry — listener casts `{Started, pid}`, conn +actor monitors-free self-deregisters via a drop guard cast. This registry +is also chunk 3's reaper and the future ws/channels introspection point, +so build it once here. + +### 3. Enforce the timeouts we advertise +**Decision fork — pick one before implementing:** + +- **(a) Reaper in urus.** The chunk-2 registry tracks per-conn + `last_activity`; a ticker (`smarm::sleep` loop) sweeps and + `request_stop`s expired conns. Works today, zero smarm changes. + Granularity = sweep interval; conns must stamp activity (one cast per + request — measurable but small). +- **(b) `wait_readable_timeout` in smarm.** A first-of(fd, timer) wait. + The v0.7 epoch-stamped consuming-unpark machinery exists precisely for + multi-arm waits (`select_timeout` is the channel-flavoured proof), but + fd arms aren't `Selectable` today — this is real smarm work in + `wait_fd`/io thread. Cleaner per-conn semantics, benefits every future + smarm io consumer, no reaper bookkeeping. + +Leaning **(a) now, (b) later**: (a) ships the security fix this cycle and +the registry exists anyway; (b) becomes a smarm roadmap item whose landing +lets the reaper's stamps go away. But if you'd rather not build throwaway: +(b) first is defensible — say the word. + +### 4. Hygiene +- Registry names: `urus.server`, `urus.listener.{n}` via `smarm::register` + (debuggability; `whereis` in tests). +- README: rewrite Roadmap section to point here; document `Handle`. +- `smarm-trace` feature passthrough already exists — verify it still + compiles under `--features smarm-trace`. + +**Exit criteria:** kill -TERM equivalent drains cleanly in the crud example; +an idle keep-alive conn is reaped at `keep_alive_timeout`; a deliberately +panicking listener restarts under load without dropped accepts (test each). + +--- + +## v0.3 — Streaming bodies + SSE + +The slipped v1 goal, and the WebSocket prerequisite (an upgraded connection +is "a response that never ends"). + +1. `RespBody::Stream(...)` — handler hands back a writer callback or a + `Receiver>`; conn actor switches to chunked transfer-encoding + (HTTP/1.1) and pumps until the stream closes. Keep-alive after a + completed chunked response. +2. Chunked **request** bodies (`parser.rs` currently returns 411) — same + cycle, the codec knowledge is fresh. +3. SSE sugar: `Conn::sse()` → an `EventSender` (`.send(event, data)`), + sets `content-type: text/event-stream`, disables buffering, heartbeats + via comment lines on a timer. An SSE conn is just a Stream body that + the reaper exempts from `request_timeout` (keep-alive ping instead). + +--- + +## v0.4 — WebSocket upgrade path + +1. **Handshake**: detect `Connection: Upgrade` + `Upgrade: websocket` in + the pipeline; `Conn::upgrade(handler)` short-circuits — conn actor + validates `Sec-WebSocket-Key`, emits 101, exits the HTTP loop. + (SHA-1 + base64 needed; vendor a tiny SHA-1 rather than pulling a + crate — urus has two deps and it should stay that way.) +2. **Frame codec**: RFC 6455 framing — fragmentation, masking (client + frames are always masked), control frames (ping/pong/close), 125-byte + control-payload rule, close-code handshake. +3. **Duplex topology** (the design decision from the assessment): + - **(a) Two actors, dup'd fd**: reader actor on the dup, writer actor + on the original, writer owns a `Receiver`; the user handler + talks to both via channels. Proven trick (listener pool), works today. + - **(b) One actor, smarm first-of(fd-readable, channel)**: needs fd + arms in select — the same smarm work as v0.2-3(b). Simpler topology, + no dup, but blocked on smarm. + Default (a); revisit if (b) lands for timeouts anyway. +4. Handler API sketch (settle in design pass): trait with + `on_message/on_close` + a `WsSender` handle — gen_server-shaped, so a + ws connection *is* an actor you can monitor, link, and register. + +## v0.5 — PubSub + +`urus::pubsub`, phoenix_pubsub-shaped, local node only (no distribution — +smarm has none): + +- `subscribe(topic)`, `unsubscribe(topic)`, `broadcast(topic, msg)`, + `broadcast_from(self, topic, msg)`. +- Topic table: gen_server owning `HashMap>)>>`; + shard N ways by topic hash if the single server measures hot (don't + pre-shard; bench first — smarm channel sends are cheap). +- Dead subscriber cleanup via `monitor` + `handle_down` (this is exactly + what 06-10's gen_server `handle_down` was built for). +- `Arc` payloads: one allocation per broadcast, not per subscriber — + the shared-heap advantage over BEAM, take it. + +Deliberately independent of HTTP — usable from any smarm app. Separate +module now, candidate for crate extraction later. + +## v0.6 — Channels + +Phoenix channels: the join/leave/event protocol over WebSocket transport, +PubSub underneath. + +- `Channel` trait: `join(topic, payload, socket)`, `handle_in(event, + payload, socket)`, `handle_out`, `terminate`. +- Topic router (`"room:*"` patterns) → channel actor per (conn, topic), + spawned under the ws connection, linked so conn death reaps channels. +- Wire format: Phoenix V2 JSON serializer + (`[join_ref, ref, topic, event, payload]`) — free interop with + phoenix.js clients is the killer feature; needs a JSON dep or a + hand-rolled mini-codec (decision point — this would be dep #3). +- Presence: explicitly out of scope until distribution exists somewhere. + +--- + +## Later / icebox + +- **HTTP/2** — spec §2.2 already designs the stream-actor demux (zero-copy + ownership transfer, conn actor owns HPACK + flow control, stream actors + run the pipeline). Big: HPACK, flow control, priorities. The Stressgrid + constraint (no stream-actor cost on HTTP/1.1) stays law. +- **SSE ergonomics round 2** — auto-reconnect support (`Last-Event-ID`), + event-id bookkeeping, backpressure policy on slow consumers. +- **TLS** — still deferred; acceptor design still must not preclude it. +- **`wait_readable_timeout` / fd-in-select upstreaming** — the smarm-side + item that simplifies v0.2-3 and v0.4-3 retroactively. +- **Middleware ecosystem** — compression, static files, auth plugs. +- **Bench suite** — `urus-bench-spec.md` exists in the artefact store; + wire it up once v0.2 lands (supervision changes the hot path not at all, + but prove it).