smarm 0.7 (415effb, "lifetime is the actor's — refs are addresses") removed the
rule these three actors were built on: a GenServerRef no longer owns the server,
the loop holds its own inbox sender, and the inbox never closes when the last ref
drops. urus's pubsub table, channel hub bus and session registries were still
governed by that deleted rule — PubSub::new() spawned the table and the handle
owned its life — so nothing commanded them to stop. What still terminated a run
was the root-exit sweep, racing the drain: shutdown_with_open_chat_terminates and
channels_wire::shutdown_with_open_channel_terminates failed 4 times in 25
--all-features runs with "serve did not return: ... outlived the drain: Timeout".
Zero in 10 full-suite runs after this change.
The fix is not a supervisor wrapped around the old shape. Every gotcha in this
area descended from constructors that spawn: PubSub::new(), ChannelHub::new() and
PrefixRouter::channel_session() all started actors, which forced in-runtime-only
construction, which forced the Arc<OnceLock<..>> lazy-init from the first handler,
which forced the "cell must not be static" and "a relay must never hold a PubSub
clone" rules. Five documented rules propping up one inverted dependency. So:
description is separated from instantiation.
- PubSub<M> is a name, not a GenServerRef: const-constructible, Copy, spawns
nothing, valid outside the runtime and in a static. Operations resolve through
the registry per call, so a table restarted by its supervisor is reached
transparently (one lookup per broadcast — bench before caching a ref, which
would go stale across exactly the restart the supervisor exists to perform).
PubSub::new() is gone; PubSub::new(name) + PubSub::child() replace it.
- ChannelHub::new(bus, router) returns (hub, Vec<ChildSpec>) — the bus table plus
one registry per session route. Returning both is the point: a hub whose
children were never started compiles and fails on the first join, so the vec is
not left behind a method you can forget to call. #[must_use].
- channel_session gains a registry name; each session registry is separately
named and separately supervised.
- serve_with/serve_with_shutdown take a Vec<ChildSpec> of app children and build
the root as RestForOne[..app children, endpoint]. They start before the
endpoint and, shutdown being ordered in reverse, stop after it has drained, so
a request still in flight can reach the bus. RestForOne because a bus crash
leaves live sockets addressing a table that no longer knows them.
- Deleted: the Arc<OnceLock> idiom from both examples and both test pipelines,
and the module rules that existed only to hand-manage a refcount.
Known cost, not fixed here: channel_session("session:*", "chat-sessions", f) puts
two unrelated string literals side by side and nothing catches a transposition —
a RegistryName newtype is the obvious follow-up.
Tests: 111 lib + 50 integration + 2 doc green, clippy clean, 10/10 full-suite
runs. Unit tests poll for name binding before use — smarm's start-order-is-not-
start-readiness gap; real apps don't hit it, since a handler only runs once a
connection has been accepted.
- crud is now the demonstrator: app owns smarm runtime + root supervisor,
store actor and urus::endpoint as ordered siblings (store first, so
reverse-order shutdown drains HTTP before stopping the store),
Shutdown::Infinity on the endpoint, shutdown via
rt.handle().request_shutdown(root_sup) from the stdin thread.
Deletes two kludges the old shape forced:
* static OnceLock<Sender> spawn-on-first-use -> a supervised child
that self-registers a typed Name; handlers use smarm::send per
request and turn 'between incarnations' into a 503 instead of
panicking on a dropped store.
* static SHUTTING_DOWN AtomicBool + 250ms recv_timeout poll in the
store loop and in the SSE ticker -> a plain park; the tree stops
both. Smoke-tested live: CRUD round-trips, SSE stream, clean drain
with the stream open, port closed after.
- Other examples stay short and on serve*, updated for the split config
(serve_with(cfg, smarm::Config, pipe) / serve_with_shutdown(..., signal)).
plain_serve's URUS_SCHED_THREADS now builds a smarm::Config.
ws_chat's doc block explains the OnceLock is a serve*-only workaround
and points at crud for the clean shape.
- README: new 'Your Own Supervision Tree' section (endpoint as the real
API), graceful shutdown reframed as the serve*-only path, Config table
loses scheduler_threads and gains name, PubSub rule 1 notes the
supervised-sibling alternative.
111 lib + 50 integration + 2 doc tests green; clippy clean.
A file-based way to set tuning knobs without recompiling. urus is a
library, so it never presumes a config path or reads the environment — the
binary hands the text in:
- Config::with_toml_str(&str) -> Result<Config, ConfigError>: sparse overlay
onto an existing Config (built with the addr the binary chose). Only keys
present are applied; durations are integer seconds; unknown keys are a
hard error (deny_unknown_fields) so a typo is loud, not a silent no-op.
- Scope: the four slowloris knobs (head_timeout_secs, body_timeout_secs,
body_burst_bytes, body_stall_timeout_secs). Migrating the rest of Config
into the file is a separate, additive job — TomlOverrides just grows.
- Feature `config-file = ["dep:serde", "dep:toml"]`; the optional serde dep
gains the derive feature. The default build is unchanged (deps + code are
all gated).
- examples/serve_toml.rs (required-features = ["config-file"]): a `--config
PATH` demo with no presumed default location. plain_serve and its env
vars are left untouched.
Tests (feature-gated): empty keeps defaults, partial overrides only named,
full overrides all, unknown key errors, malformed errors. 84 lib with the
feature / 79 without; clippy --lib clean both ways; e2e smoke serves 200
from a file and rejects an unknown key loudly.
E1 (herd-vs-stranding discrimination) sweeps smarm scheduler count at
fixed low concurrency. Strict parse — a malformed value panics instead
of silently reverting to default, and the effective value is logged to
stderr so every bench cell's server.log records it (same per-cell
verification discipline as mode-verify).
The jar's ka-vs-close throughput inversion was observed on the box but its
run configuration died with the job workspaces, so it can't be re-derived —
this commit makes the comparison a committed, controlled experiment instead
of a lost one-off.
- examples/plain_serve: the load_profile request path with zero causal
machinery (same route/handler/Config), serving until killed. The plain
half of the A/B; throughput measured externally by wrk.
- scripts/ka-close-matrix.sh: {ka,close} x {plain,causal} on fresh ports,
byte-identical wrk args per mode pair except the Connection: close
header, disjoint-core pinning for server vs wrk, per-cell curl
verification of the negotiated connection behavior, ss/env capture, and
a parsed summary with a close/ka ratio verdict. The causal cells double
as the forgiveness-fix box validation (v13 pending item, pull-forward
agreed): key on forgiven staying ~ injected x parked-actors in
close-mode churn, not process-lifetime phantoms.
Server-only discovery tool: real hot path, no planted bottleneck, no
in-process loadgen. wrk (or any external generator, pinned to other
cores) drives GET /json/:id; the sweep runs the causal_bench plan over
the five lib sites. Trust gates only (traffic flowed, ledger printed) —
no known-answer verdict, this one asks the question instead.
examples/causal_bench.rs (required-features smarm-causal): the Phase 2
RFC 007 test case — a real urus server replacing the synthetic demo as
validation workload.
Shape: 16 plain-OS-thread clients hammer GET /order/:id over blocking
loopback keep-alive TCP (OS threads can't absorb injected virtual delay
— the established trick, so only server code is delayable). The handler
calls a single store actor (crud's once-cell pattern, serialization
structural) burning 400µs of calibrated *work* under
causal_site!("store") — work-shaped, not timed, or injection reads as a
no-op. 50µs handler render lands under the enclosing pipeline site.
Known answer: store serialized + saturated => ceiling 2500 rps; speedup
p => x1/(1-p): +33% @25, +100% @50. The five lib sites are tens of µs
and parallel across conns => ~0. Verdict mirrors the demo: store @25 >
+15%, others < +10%, SKIPPED under 4 cores; magnitudes trusted to ~±15%
per the smarm-side validation notes, ranking is the hard check.
1-core sandbox smoke (verdict SKIPPED but numbers indicative): store
+26.2% @25 / +74.0% @50, all other sites within ±2%. Unlike the demo
this workload keeps its bottleneck structure on one core — everything
but the store is IO-parked — and the theory shortfall there is plain
core contention (render/parse share the store's CPU), which the 24-core
sweep should close.
Pre-existing, surfaced by the toolchain bump (rust-version is 1.95; the
sandbox gates with stable 1.97). All four are cargo clippy --fix output
with the mechanical-collapse indentation hand-tidied to house style;
the parser change is semantics-preserving (a non-100-continue Expect
value now falls to the _ arm instead of an empty if — headers.append
still runs after the match either way). No fmt pass: rustfmt would
clobber the aligned-assignment style, so only the touched lines moved.
examples/channels_chat.rs (required-features phoenix): ephemeral
room:* and persistent session:* side by side over phoenix.js V2 JSON,
stdin-driven graceful shutdown (the ws_chat pattern). Smoke-tested
live: rejoin after transport drop lands on the same instance (state
counter persists across three transports), broadcasts carry the new
join generation's ref.
README: channels section (feature split, core trait walkthrough,
session persistence semantics incl. the re-entrant-join contract).
ROADMAP: v0.6 marked done with the full as-landed decision list,
chunked as committed.
WsHandler::on_open(&mut self, sender) — defaulted, non-breaking,
added mid-chunk for veto-by-diff (the v0.3 "push" precedent for
defaults-level decisions): without it a listen-only ws client can
never be subscribed, since on_message never fires for a client that
doesn't send. Runs exactly once, first, in the conn actor, before any
buffered pipelined frame is decoded. Panic contract identical to
on_message: check_cancelled re-raise dance, else 1011 and no
on_close.
DISCOVERY 1 (documented, not fixed — inherent): the 101 reaches the
client BEFORE on_open runs, so "is my subscription live yet" is a
race client-side. App-level ack is the answer (any reply proves
on_open completed; callbacks are sequential). The integration tests
hit this immediately (first read of a join notice flaked into a 5s
read-timeout) and use a sync/synced ack; same reason Phoenix joins
reply.
DISCOVERY 2 (the structural one): PubSub::new() is in-runtime only,
but pipelines are built pre-runtime. crud's static-OnceLock bootstrap
DOES NOT COMPOSE with serve_with_shutdown here: a static pins the
table actor's last ServerRef forever, its inbox never closes, the
gen_server never exits, AllDone is unreachable — serve hangs (the
cross-thread-unpark limitation closes the workaround routes). The
pattern that works: NON-static Arc<OnceLock<PubSub<M>>> captured by
the route closure. Drain stops conns+listeners -> last Arc<Pipeline>
drops IN-runtime -> cell+handle drop -> inbox closes -> table exits.
Corollary: relays hold the Receiver only, NEVER a PubSub clone
(relay holds table's inbox open, table holds relay's receiver open:
mutual keepalive, shutdown hangs). Both rules in module docs, README,
and enforced by the new shutdown_with_open_chat_terminates test:
two open chat sockets + live relays + live table, shutdown must
return within 5s.
examples/ws_chat.rs: rooms as topics (room:{name}), on_open
subscribes (conn-actor pid: the monitor scopes cleanup to the
session) + spawns the relay (rx -> WsSender clone; exits on prune or
WsClosed), on_message broadcast_from(self_pid()) so the speaker is
never echoed, on_close broadcasts the leave notice and relies on the
monitor for unsubscribe.
Integration: ws_chat_broadcast_reaches_other_client_not_sender
(no-echo proven orderingly, no timeout reads: B speaks after A, A's
next frame must be B's) + shutdown_with_open_chat_terminates.
hammer.sh default subset now includes ws_ (v0.4 ran it ad hoc; ws IS
conn-lifecycle). Hammered: 35x subset (incl. both new tests) + 3x
full + 1x full under smarm-trace, all green. dbg!-grep clean. smarm
PRISTINE. README pub/sub + on_open sections; ROADMAP v0.5 as-landed.
Suite: 67 unit + 42 integration + 2 doc.
Topology (b) as agreed: ONE actor per connection via smarm RFC 008 fd
arms. After the 101, ws::duplex::run_duplex replaces the HTTP loop:
try_select(&[&outbound_rx, &FdArm::readable(fd)]) per iteration,
outbound at index 0 (priority order = owed writes drain before reads;
honest backpressure). Accepted gap documented: mid-write of a large
outbound frame the actor isn't reading.
Handler API (deferred questions, as answered this session):
- WsHandler { on_message, on_close } runs IN the conn actor's select
loop; concurrency = spawn an actor with a WsSender clone (the SSE
producer pattern). Conn::upgrade(self, handler) flat signature;
WsUpgrade grows from marker to Box<dyn WsHandler> payload.
- WsSender: Clone; send/text/binary/ping/close -> Err(WsClosed).
Unbounded channel like SSE; slow client bounded by write_timeout.
- Control frames invisible v1: auto-pong inline (5.5.2), peer close
auto-echoed (5.5.1; server closes TCP first per 7.1.1) surfacing
only as on_close(Some(code), reason); wordless endings (EOF, write
failure, drain timeout) = on_close(None, ""). on_ping/on_pong can
land later as default methods, non-breaking.
- Server-initiated close (WsSender::close, FrameError->1002/1009/1007,
handler panic->1011): close out, bounded drain (one write_timeout
budget) for the peer echo, data discarded (1.4). Handler panics
re-raise smarm's stop sentinel first (the pipeline catch_unwind
dance, replicated).
- Caps: Config.max_frame_payload (1 MiB) / max_message_bytes (4 MiB).
Conn actor: 101 branch now hands fd + leftover buf (pipelined first
frame carries over; tested) + boxed handler to run_duplex; registry
entry stays Busy for the ws lifetime, so graceful shutdown force-stops
the conn out of the select park at the drain deadline (tested — the
in-runtime request_stop wake covers select parks too).
Tests: chunk-1 EOF test rewritten into a 9-test duplex suite (echo,
pipelined first frame, ping/pong, both close directions, 1002 unmasked,
1009 header-cap, fragmentation with interleaved ping, shutdown
force-stop). Suite 59u+40i+2d. Hammer: 35x lifecycle+ws subset + 3 full
+ 1 full under smarm-trace, all green; no AlreadyExists out of
try_select (the fresh eager-cleanup path held). Validated against
python websocket-client (echo, pong payload, close 1000).
examples/ws_echo.rs: /echo in-actor + /clock producer-spawning.
README: write_timeout in the Config sample, read/write timeout-split
semantics spelled out, new Streaming Responses and Server-Sent Events
sections (producer contract, 1.0 fallback, heartbeat liveness), roadmap
summary bumped. ROADMAP: v0.3 marked done with the as-landed decisions
(pull shape per the user's fork answer; Q3 timeout split as proposed)
and verified exit criteria. crud example: GET /ticker SSE route — one
event/second, producer exits on SseClosed or the example's shutdown
flag (so an open ticker doesn't block AllDone past the drain).
Two wall-clock budgets in the conn read path, both via
smarm::wait_readable_timeout:
- keep_alive_timeout: idle budget between requests — how long we park
waiting for the FIRST byte of a request (including the first request on
a fresh connection). Expiry closes silently; nothing is owed to a
client that isn't talking.
- request_timeout: per-request budget as an Instant deadline from the
first byte of a request until the request (head + body) is fully read.
Pipelined leftovers in the buffer at cycle start count as a started
request. The deadline is returned from read_head and threaded into
read_body so head and body share one budget. Pipeline run time is NOT
covered. Each read wait uses whichever budget is currently active;
deadlines are Instants, so EAGAIN retries don't reset the clock.
Expiry mid-head gets a best-effort 408 via try_write_once: a single
non-parking write syscall — a client that stalls its read side must not
defeat the timeout by making the 408 write park forever. Expiry mid-body
just closes.
examples/crud.rs gains stdin-Enter graceful shutdown (plain OS thread on
read_line -> handle.shutdown(); no signal crate). Doing so surfaced a
pattern worth knowing: crud's lazily-spawned store actor parked forever
in recv(), which blocks smarm's AllDone, so serve_with_shutdown never
returned (gdb: scheduler idle in poll_wake, store the only live actor).
It can't be messaged awake from the stdin thread either — a cross-thread
send's unpark is a no-op without runtime TLS, the same limitation behind
SHUTDOWN_POLL. Fix: store_loop recv_timeout(250ms) + a SHUTTING_DOWN
AtomicBool set by the stdin thread before handle.shutdown(). This poll
dies with the cross-thread-unpark limitation; recorded in smarm docs.
Tests: idle keep-alive conn reaped at a small keep_alive_timeout;
slowloris partial-head stall killed at request_timeout with the 408
observed. Timeout+shutdown subset hammered 30x clean; full suite 3x;
smarm-trace build and suite clean.
Claude
Markk116