feat(ws): duplex wiring + handler API + echo example (v0.4 chunk 3)

Topology (b) as agreed: ONE actor per connection via smarm RFC 008 fd
arms. After the 101, ws::duplex::run_duplex replaces the HTTP loop:
try_select(&[&outbound_rx, &FdArm::readable(fd)]) per iteration,
outbound at index 0 (priority order = owed writes drain before reads;
honest backpressure). Accepted gap documented: mid-write of a large
outbound frame the actor isn't reading.

Handler API (deferred questions, as answered this session):
- WsHandler { on_message, on_close } runs IN the conn actor's select
  loop; concurrency = spawn an actor with a WsSender clone (the SSE
  producer pattern). Conn::upgrade(self, handler) flat signature;
  WsUpgrade grows from marker to Box<dyn WsHandler> payload.
- WsSender: Clone; send/text/binary/ping/close -> Err(WsClosed).
  Unbounded channel like SSE; slow client bounded by write_timeout.
- Control frames invisible v1: auto-pong inline (5.5.2), peer close
  auto-echoed (5.5.1; server closes TCP first per 7.1.1) surfacing
  only as on_close(Some(code), reason); wordless endings (EOF, write
  failure, drain timeout) = on_close(None, ""). on_ping/on_pong can
  land later as default methods, non-breaking.
- Server-initiated close (WsSender::close, FrameError->1002/1009/1007,
  handler panic->1011): close out, bounded drain (one write_timeout
  budget) for the peer echo, data discarded (1.4). Handler panics
  re-raise smarm's stop sentinel first (the pipeline catch_unwind
  dance, replicated).
- Caps: Config.max_frame_payload (1 MiB) / max_message_bytes (4 MiB).

Conn actor: 101 branch now hands fd + leftover buf (pipelined first
frame carries over; tested) + boxed handler to run_duplex; registry
entry stays Busy for the ws lifetime, so graceful shutdown force-stops
the conn out of the select park at the drain deadline (tested — the
in-runtime request_stop wake covers select parks too).

Tests: chunk-1 EOF test rewritten into a 9-test duplex suite (echo,
pipelined first frame, ping/pong, both close directions, 1002 unmasked,
1009 header-cap, fragmentation with interleaved ping, shutdown
force-stop). Suite 59u+40i+2d. Hammer: 35x lifecycle+ws subset + 3 full
+ 1 full under smarm-trace, all green; no AlreadyExists out of
try_select (the fresh eager-cleanup path held). Validated against
python websocket-client (echo, pong payload, close 1000).
examples/ws_echo.rs: /echo in-actor + /clock producer-spawning.
This commit is contained in:
Claude
2026-06-12 09:44:35 +00:00
parent 64137cccf0
commit ffc579c500
10 changed files with 902 additions and 116 deletions
+15 -9
View File
@@ -366,25 +366,31 @@ impl Conn {
self
}
/// Accept a WebSocket upgrade on this request (RFC 6455 §4.2).
/// Accept a WebSocket upgrade on this request (RFC 6455 §4.2),
/// handing `handler` to the duplex loop that takes the socket over.
///
/// On a valid handshake: status `101`, the `upgrade`/`connection`/
/// `sec-websocket-accept` response headers, the [`WsUpgrade`] marker
/// the connection actor acts on, and the pipeline is halted.
/// `sec-websocket-accept` response headers, the [`WsUpgrade`] payload
/// the connection actor acts on, and the pipeline is halted. After
/// the 101 the connection actor leaves HTTP and drives `handler` —
/// see [`WsHandler`](crate::ws::WsHandler) for the callback contract.
///
/// On an invalid handshake: the appropriate rejection (`400`, or
/// `426` + `sec-websocket-version: 13` for a version mismatch),
/// empty body, pipeline halted; the connection stays plain HTTP with
/// normal keep-alive semantics. Pre-check with
/// On an invalid handshake: `handler` is dropped untouched and the
/// appropriate rejection goes out (`400`, or `426` +
/// `sec-websocket-version: 13` for a version mismatch), empty body,
/// pipeline halted; the connection stays plain HTTP with normal
/// keep-alive semantics. Pre-check with
/// [`ws::handshake::is_upgrade_request`](crate::ws::handshake::is_upgrade_request)
/// to branch before committing.
///
/// [`WsUpgrade`]: crate::ws::WsUpgrade
pub fn upgrade(mut self) -> Self {
pub fn upgrade(mut self, handler: impl crate::ws::WsHandler) -> Self {
use crate::ws::{handshake, Rejection};
match handshake::validate(&self) {
Ok(accept) => {
self.upgrade = Some(crate::ws::WsUpgrade { _priv: () });
self.upgrade = Some(crate::ws::WsUpgrade {
handler: Box::new(handler),
});
self.put_status(101)
.put_header("upgrade", "websocket")
.put_header("connection", "Upgrade")
+28 -11
View File
@@ -59,6 +59,13 @@ pub struct ConnLimits {
/// A client that stops reading mid-response is dropped when its
/// socket buffer fills and a write stalls past the budget.
pub write_timeout: Duration,
/// WebSocket: hard cap on a single frame's payload, enforced from
/// the frame header BEFORE the payload is buffered. Violation closes
/// with 1009.
pub max_frame_payload: usize,
/// WebSocket: hard cap on a complete (reassembled) message; spans
/// fragments. Violation closes with 1009.
pub max_message_bytes: usize,
}
impl Default for ConnLimits {
@@ -71,6 +78,8 @@ impl Default for ConnLimits {
keep_alive_timeout: Duration::from_secs(60),
request_timeout: Duration::from_secs(30),
write_timeout: Duration::from_secs(30),
max_frame_payload: 1024 * 1024,
max_message_bytes: 4 * 1024 * 1024,
}
}
}
@@ -231,17 +240,25 @@ pub fn run_connection(
.put_body(RespBody::Empty);
}
// ----- 3.5 WebSocket upgrade short-circuit. -----
// An accepted handshake (marker + 101) ends HTTP on this socket:
// write the 101 head and leave the request loop. Chunk 3 (duplex
// wiring) takes the socket over right here; until then leaving
// the loop closes it via OwnedFd::drop — the 101 is still the
// honest, testable wire artefact. The status check is defensive:
// a post-handler plug that clobbered the 101 forfeits the
// upgrade and falls through to plain HTTP.
// ----- 3.5 WebSocket upgrade. -----
// An accepted handshake (payload + 101) ends HTTP on this socket:
// write the 101 head, hand the fd to the duplex loop with the
// boxed handler and any bytes already read past this request (a
// client may pipeline its first frame behind the handshake — those
// bytes are ws bytes now). The registry entry stays Busy for the
// whole ws lifetime: an open WebSocket is in-flight work, not
// reapable idle HTTP; graceful shutdown force-stops it out of the
// select park at the drain deadline. The status check is
// defensive: a post-handler plug that clobbered the 101 forfeits
// the upgrade and falls through to plain HTTP.
if response_conn.upgrade.is_some() && response_conn.status == Some(101) {
let head = parser::serialise_response(&response_conn, true);
let _ = write_all(raw, &head, Instant::now() + limits.write_timeout);
if write_all(raw, &head, Instant::now() + limits.write_timeout).is_err() {
return;
}
let upgrade = response_conn.upgrade.take().expect("checked above");
buf.drain(..head_len + consumed_past_head);
crate::ws::duplex::run_duplex(raw, buf, upgrade.handler, &limits);
return;
}
@@ -550,7 +567,7 @@ fn read_chunked_body(
// `ErrorKind::TimedOut` when `deadline` passes before the fd turns
// readable, or the last io error.
fn read_some(
pub(crate) fn read_some(
fd: RawFd,
buf: &mut Vec<u8>,
chunk: usize,
@@ -616,7 +633,7 @@ fn try_write_once(fd: RawFd, buf: &[u8]) {
// are down — a client that stops reading must not pin this actor in
// wait_writable forever (the write-side twin of slowloris).
fn write_all(fd: RawFd, mut buf: &[u8], deadline: Instant) -> io::Result<()> {
pub(crate) fn write_all(fd: RawFd, mut buf: &[u8], deadline: Instant) -> io::Result<()> {
while !buf.is_empty() {
// Park on writability before each syscall, bounded by the budget.
let remaining = deadline.saturating_duration_since(Instant::now());
+1
View File
@@ -34,6 +34,7 @@ pub use conn::{Assigns, Body, Conn, HeaderMap, HttpVersion, Method, Params, Resp
pub use plug::{Next, Pipeline, Plug};
pub use router::Router;
pub use sse::{EventSender, SseClosed};
pub use ws::{Message, WsClosed, WsHandler, WsSender};
pub use serve::{
serve, serve_with, serve_with_shutdown, shutdown_handle, Config, Handle, ShutdownSignal,
};
+10
View File
@@ -47,6 +47,12 @@ pub struct Config {
/// connections are closed immediately on shutdown and do not run the
/// clock out.
pub drain_timeout: Duration,
/// WebSocket: cap on a single frame's payload (header-checked
/// before buffering; violation closes 1009).
pub max_frame_payload: usize,
/// WebSocket: cap on a complete reassembled message (spans
/// fragments; violation closes 1009).
pub max_message_bytes: usize,
/// Number of smarm scheduler OS threads. `None` means smarm's default
/// (one per CPU). Set this to a small fixed number in tests so multiple
/// concurrent test servers don't oversubscribe the host.
@@ -69,6 +75,8 @@ impl Config {
write_timeout: Duration::from_secs(30),
max_body_bytes: 16 * 1024 * 1024,
drain_timeout: Duration::from_secs(30),
max_frame_payload: 1024 * 1024,
max_message_bytes: 4 * 1024 * 1024,
scheduler_threads: None,
}
}
@@ -82,6 +90,8 @@ impl Config {
keep_alive_timeout: self.keep_alive_timeout,
request_timeout: self.request_timeout,
write_timeout: self.write_timeout,
max_frame_payload: self.max_frame_payload,
max_message_bytes: self.max_message_bytes,
}
}
}
+398
View File
@@ -0,0 +1,398 @@
//! The WebSocket duplex loop — v0.4 chunk 3.
//!
//! After an accepted upgrade the connection actor calls [`run_duplex`],
//! which replaces the HTTP request loop for the rest of the socket's
//! life. ONE actor speaks both directions (the v0.4 topology decision):
//! each iteration parks on a two-arm select —
//!
//! try_select(&[&outbound_rx, &FdArm::readable(fd)])
//!
//! — smarm's RFC 008 fd arms composed with a channel arm on one wait
//! epoch; urus is deliberately the first consumer of that machinery.
//! The outbound arm sits at index 0 (select is priority-ordered): owed
//! writes drain before we read more, which is honest backpressure.
//! Accepted gap, by design: while a large outbound frame is mid-write
//! the actor isn't reading, so a ping arriving during that write is
//! answered after it completes.
//!
//! Handler model (the v0.4 chunk-3 decisions, mirroring SSE):
//! - [`WsHandler`] callbacks run IN the connection actor, inside the
//! select loop. A slow `on_message` stops reads — backpressure, not a
//! bug. Handlers needing concurrency spawn their own actor and hand it
//! a [`WsSender`] clone, exactly like an SSE producer.
//! - [`WsSender`] is the clonable outbound handle; every method returns
//! `Err(WsClosed)` once the connection is gone.
//! - Control frames are invisible to the handler in v1: pings are
//! auto-ponged (RFC 6455 §5.5.2 MUST), pongs are absorbed, a peer
//! close is auto-echoed (§5.5.1) and surfaces only as `on_close`.
//! `on_ping`/`on_pong` hooks can land later as default trait methods
//! without breaking anyone.
//!
//! Liveness mirrors SSE: there is no request clock on an open
//! WebSocket. An idle connection parks in the select (stoppable by a
//! draining registry — graceful shutdown force-stops it at the drain
//! deadline); a dead client is caught when a write stalls past
//! `write_timeout`. Reads have no deadline of their own — the select
//! only wakes the read path when bytes are pending.
use crate::conn_actor::{read_some, write_all, ConnLimits};
use crate::ws::frame::{self, Assembler, Frame, FrameError, Message, Opcode};
use smarm::FdArm;
use std::os::fd::RawFd;
use std::time::Instant;
// ---------------------------------------------------------------------------
// Handler trait
// ---------------------------------------------------------------------------
/// Application callbacks for an upgraded WebSocket connection. Both run
/// inside the connection actor's select loop — see the module docs for
/// what that implies (backpressure, spawn-for-concurrency).
///
/// A panic in `on_message` closes the connection with a `1011 internal
/// error` close frame and skips `on_close` (the handler is not re-entered
/// after it panicked).
pub trait WsHandler: Send + 'static {
/// A complete data message (fragments already reassembled, text
/// already UTF-8 validated) arrived from the client.
fn on_message(&mut self, msg: Message, sender: &WsSender);
/// The connection is over. `code`/`reason` are what the PEER said:
/// `Some` iff a close frame was received from the client (whether it
/// initiated or echoed ours); `None` for everything wordless — EOF,
/// write failure, or a close handshake that timed out. Called exactly
/// once, last, on every exit path except a handler panic or a
/// force-stop unwind.
fn on_close(&mut self, code: Option<u16>, reason: &str) {
let _ = (code, reason);
}
}
// ---------------------------------------------------------------------------
// WsSender — the clonable outbound handle
// ---------------------------------------------------------------------------
/// The connection is gone (client disconnect, write timeout, close
/// handshake completed, or server shutdown). Producers should exit when
/// they see this — the exact contract of `SseClosed`.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct WsClosed;
impl std::fmt::Display for WsClosed {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("WebSocket connection closed")
}
}
impl std::error::Error for WsClosed {}
pub(crate) enum Outbound {
Msg(Message),
Ping,
Close(u16, String),
}
/// Outbound handle for an upgraded connection. Clonable — hand copies to
/// producer actors freely; the connection outlives none of them (sends
/// after it ends return `Err(WsClosed)`).
///
/// v1 sends each message as a single unfragmented frame; outbound
/// fragmentation is not implemented.
#[derive(Clone)]
pub struct WsSender {
pub(crate) tx: smarm::Sender<Outbound>,
}
impl WsSender {
/// Queue a data message for the client.
pub fn send(&self, msg: Message) -> Result<(), WsClosed> {
self.tx.send(Outbound::Msg(msg)).map_err(|_| WsClosed)
}
/// Sugar: queue a text message.
pub fn text(&self, s: impl Into<String>) -> Result<(), WsClosed> {
self.send(Message::Text(s.into()))
}
/// Sugar: queue a binary message.
pub fn binary(&self, b: impl Into<Vec<u8>>) -> Result<(), WsClosed> {
self.send(Message::Binary(b.into()))
}
/// Queue a ping (empty payload). Application-level liveness beyond
/// the built-in write-timeout detection; the pong is absorbed by the
/// loop in v1.
pub fn ping(&self) -> Result<(), WsClosed> {
self.tx.send(Outbound::Ping).map_err(|_| WsClosed)
}
/// Initiate the close handshake: a close frame with `code`/`reason`
/// goes out, the loop then waits (bounded by `write_timeout`) for the
/// peer's echo before dropping the socket. `reason` is truncated to
/// fit the 125-byte control payload on a char boundary.
pub fn close(&self, code: u16, reason: &str) -> Result<(), WsClosed> {
self.tx
.send(Outbound::Close(code, reason.to_string()))
.map_err(|_| WsClosed)
}
}
// ---------------------------------------------------------------------------
// The loop
// ---------------------------------------------------------------------------
/// Run the duplex loop until the connection ends. `buf` carries any bytes
/// already read past the upgrade request's head — a client may pipeline
/// its first frame behind the handshake, and those bytes belong to us
/// now.
pub(crate) fn run_duplex(
raw: RawFd,
mut buf: Vec<u8>,
mut handler: Box<dyn WsHandler>,
limits: &ConnLimits,
) {
let (tx, rx) = smarm::channel::<Outbound>();
// We hold this sender for handler callbacks, so the rx arm can never
// report closed while the loop runs — no closed-arm starvation case.
let sender = WsSender { tx };
let mut asm = Assembler::new(limits.max_message_bytes);
loop {
// ----- 1. Decode everything already buffered. -----
// Runs before the first select so a pipelined first frame is
// served without waiting for fresh readability.
loop {
let (frame, consumed) =
match frame::decode(&buf, true, limits.max_frame_payload) {
Ok(Some(hit)) => hit,
Ok(None) => break, // incomplete; need more bytes
Err(e) => {
let peer = close_and_drain(raw, e, &mut buf, limits);
finish(&mut handler, peer);
return;
}
};
buf.drain(..consumed);
match frame.opcode {
Opcode::Ping => {
// §5.5.2 MUST pong, payload echoed. Written inline —
// "as soon as practical" beats channel ordering.
let pong = Frame::new(Opcode::Pong, frame.payload);
if write_frame(raw, &pong, limits).is_err() {
finish(&mut handler, None);
return;
}
}
Opcode::Pong => {} // invisible in v1
Opcode::Close => {
// Peer-initiated close: echo its status code (§5.5.1)
// and end. The server side closes the TCP connection
// first by design (§7.1.1 puts that burden on us).
let echo_payload =
frame.payload.get(..2).map(<[u8]>::to_vec).unwrap_or_default();
let peer = match frame::parse_close_payload(&frame.payload) {
Ok(cp) => {
let _ = write_frame(
raw,
&Frame::new(Opcode::Close, echo_payload),
limits,
);
cp
}
Err(e) => {
// Malformed close payload: answer with the
// mapped code instead of an echo.
let _ = write_frame(
raw,
&Frame::new(
Opcode::Close,
frame::close_payload(e.close_code(), ""),
),
limits,
);
None
}
};
finish_with(&mut handler, peer);
return;
}
_ => match asm.push(frame) {
Ok(Some(msg)) => {
// A panicking handler must not swallow smarm's stop
// sentinel — same re-raise dance as the pipeline.
let r = std::panic::catch_unwind(
std::panic::AssertUnwindSafe(|| {
handler.on_message(msg, &sender)
}),
);
if r.is_err() {
smarm::preempt::check_cancelled();
// Genuine handler panic: 1011 out, no further
// handler calls (no on_close).
let _ = close_and_drain_code(
raw, 1011, "", &mut buf, limits,
);
return;
}
}
Ok(None) => {} // mid-fragmentation
Err(e) => {
let peer = close_and_drain(raw, e, &mut buf, limits);
finish(&mut handler, peer);
return;
}
},
}
}
// ----- 2. Park on first-of outbound / fd-readable. -----
// Outbound at index 0: select is priority-ordered, so owed writes
// go out before we read more. try_select over select: an fd-arm
// registration failure (EBADF after a peer RST, EMFILE on the
// epoll set) is a connection event, not a crash. NOTE from the
// RFC 008 review: an `AlreadyExists` here would mean a stale
// waiter survived select's eager cleanup — that is a smarm bug
// candidate; capture it with --features smarm-trace and report,
// do not paper over.
let fd_arm = FdArm::readable(raw);
let idx = match smarm::try_select(&[&rx, &fd_arm]) {
Ok(i) => i,
Err(_) => {
finish(&mut handler, None);
return;
}
};
if idx == 0 {
// Outbound arm won. Single receiver + we hold a sender: a
// ready arm means a message is queued (Empty/closed are
// impossible here, but stay defensive).
let Ok(Some(out)) = rx.try_recv() else { continue };
match out {
Outbound::Msg(msg) => {
let f = match msg {
Message::Text(s) => Frame::new(Opcode::Text, s.into_bytes()),
Message::Binary(b) => Frame::new(Opcode::Binary, b),
};
if write_frame(raw, &f, limits).is_err() {
finish(&mut handler, None);
return;
}
}
Outbound::Ping => {
let f = Frame::new(Opcode::Ping, Vec::new());
if write_frame(raw, &f, limits).is_err() {
finish(&mut handler, None);
return;
}
}
Outbound::Close(code, reason) => {
// We initiate: close out, bounded wait for the echo.
let peer = close_and_drain_code(raw, code, &reason, &mut buf, limits);
finish(&mut handler, peer);
return;
}
}
} else {
// Readable. The deadline is nominal — the fd already reported
// ready, this returns without a fresh park in practice.
match read_some(
raw,
&mut buf,
limits.initial_read_buf,
Instant::now() + limits.write_timeout,
) {
Ok(0) => {
// EOF without a close frame: abnormal but common.
finish(&mut handler, None);
return;
}
Ok(_) => {}
Err(_) => {
finish(&mut handler, None);
return;
}
}
}
}
}
/// `on_close(None, "")` — wordless endings (EOF, write failure, errors).
fn finish(handler: &mut Box<dyn WsHandler>, peer: Option<(u16, String)>) {
finish_with(handler, peer);
}
/// `on_close` with whatever the peer said, panic-shielded like
/// `on_message` (stop sentinel re-raised; a panic here just ends the
/// already-ending connection).
fn finish_with(handler: &mut Box<dyn WsHandler>, peer: Option<(u16, String)>) {
let r = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
match &peer {
Some((code, reason)) => handler.on_close(Some(*code), reason),
None => handler.on_close(None, ""),
}
}));
if r.is_err() {
smarm::preempt::check_cancelled();
}
}
/// One frame onto the wire under a fresh `write_timeout` budget — the
/// streaming-chunk discipline, verbatim.
fn write_frame(raw: RawFd, f: &Frame, limits: &ConnLimits) -> std::io::Result<()> {
write_all(raw, &f.encode(), Instant::now() + limits.write_timeout)
}
/// Server-initiated close for a frame/assembly error: map to the §7.4
/// code and run the handshake tail.
fn close_and_drain(
raw: RawFd,
e: FrameError,
buf: &mut Vec<u8>,
limits: &ConnLimits,
) -> Option<(u16, String)> {
close_and_drain_code(raw, e.close_code(), "", buf, limits)
}
/// Write our close frame, then wait — bounded by ONE `write_timeout`
/// budget — for the peer's close frame, discarding data frames (§1.4: an
/// endpoint that has sent close may discard incoming data). Returns what
/// the peer's close said, `None` if it never arrived (EOF, timeout,
/// garbage). The socket drops at the caller either way.
fn close_and_drain_code(
raw: RawFd,
code: u16,
reason: &str,
buf: &mut Vec<u8>,
limits: &ConnLimits,
) -> Option<(u16, String)> {
let ours = Frame::new(Opcode::Close, frame::close_payload(code, reason));
if write_frame(raw, &ours, limits).is_err() {
return None;
}
let deadline = Instant::now() + limits.write_timeout;
loop {
loop {
match frame::decode(buf, true, limits.max_frame_payload) {
Ok(Some((frame, consumed))) => {
buf.drain(..consumed);
if frame.opcode == Opcode::Close {
return frame::parse_close_payload(&frame.payload)
.ok()
.flatten();
}
// Data/ping mid-handshake: discarded.
}
Ok(None) => break,
Err(_) => return None, // garbage during the tail: give up
}
}
match read_some(raw, buf, limits.initial_read_buf, deadline) {
Ok(0) | Err(_) => return None,
Ok(_) => {}
}
}
}
+13 -6
View File
@@ -12,17 +12,24 @@
//! vendoring. Base64 here is the 20-byte *encode* direction only and is
//! not cryptographic, so that stays in-tree (`handshake::b64`).
pub mod duplex;
pub mod frame;
pub mod handshake;
pub use duplex::{WsClosed, WsHandler, WsSender};
pub use frame::{Frame, FrameError, Message, Opcode};
pub use handshake::Rejection;
/// Marker carried on a [`Conn`](crate::Conn) whose handler accepted a
/// WebSocket upgrade. Opaque: chunk 3 grows this into the handler/duplex
/// handoff payload, so nothing outside the crate should depend on its
/// shape.
#[derive(Debug)]
/// Payload carried on a [`Conn`](crate::Conn) whose handler accepted a
/// WebSocket upgrade: the boxed [`WsHandler`] the duplex loop will drive
/// (the chunk-3 growth this was reserved for). Still opaque outside the
/// crate.
pub struct WsUpgrade {
pub(crate) _priv: (),
pub(crate) handler: Box<dyn WsHandler>,
}
impl std::fmt::Debug for WsUpgrade {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("WsUpgrade { handler: .. }")
}
}