Files
smarm/tests
claude-asm-audit 4c0e42152f feat(cluster): RFC 010 c10–c16, follow-ups and Phase 6 (squash of 16ef583..d9c62a8)
Tree snapshot of d9c62a8 (2026-08-18). The 20 source commits between
16ef583 (c9) and d9c62a8 were never pushed and the clone that held them
was lost; this commit carries their combined tree verbatim so the build
history stays auditable from the c1–c9 commits below it. Original
hashes as recorded in the session handoff:

  c10  f03e94d  pid targeting + auto-serialization (RemotePid, D14 name
                on the wire); Phase 3 gate
  c11  7ef4bad  DownReason::Disconnected, wire tag 5
  c12  d124162  remote monitors (Monitor/Demonitor/Down frames)
  c13  9de967b  connection-loss synthesis (A+B: Monitors::teardown +
                unread-command Disconnected); Phase 4 gate
  c14  7e822b7  eager pg eviction (reaper actor, ReaperInboxes)
       dbe1a22  InboundVerdict::label(), trace::Event::ClusterInbound
       31a9877  tests/channel.rs monitor-churn target gated on `go`
       653559e  Discovery::Withdrawn{name, addr}
  c15  b41d76e  distributed pg: Sync on NodeUp, Join/Leave broadcast,
                NodeDown sweep, members_all; PgMsg wire type
  c16  fafa881  pick_any / dispatch_any; Phase 5 complete
  Phase 6 Tier A:
       195c73e  p4  NodeEvent::NodeDown(NodeInfo)
       48fd766  p1  connector Candidate{name, addr, state}
       ce8cf99  p2+p7 conn.rs select arms as Vec<Arm>; Outbound::Drained
       bf24988  p6  RemotePid::from_local -> Option
       9ae0380  p3  PeerStanding{Free, Claimed, Dialing}
       c7d62a1  p11 cluster::Timing knobs, threaded by value
       46f171d  p11 cluster_disconnect un-ignored on SMARM_FAST_TIMING
  Phase 6 Tier B:
       391a9ae  p5  cluster::RemoteDownReason{Local, Disconnected};
                    DownReason::Disconnected removed from core
       7ddd908  p9  pg ctl channel unconditional, one cfg seam at spawn
       d9c62a8      PeerNameMismatch parks the candidate; ClusterDial trace

Verified at d9c62a8: default 361/0, cluster 448/0, clippy --lib on
default / cluster / cluster+smarm-trace, fmt, 10x flake on
cluster_dial_mismatch, 5x on cluster_pg.
2026-08-18 16:00:00 +02:00
..
2026-05-23 16:09:35 +00:00

Tests

Integration tests for the runtime. Each file owns one feature area or one class of bug. Everything here runs under plain cargo test; the loom model tests are the exception — they live in the library (src/slot_state.rs, src/run_queue.rs), not in this directory, because loom must compile the production code with shimmed atomics (see "Loom" below).

Running

cargo test                # debug build — RUN THIS ONE: all invariant asserts live
cargo test --release      # what users actually execute (LTO, no debug_asserts)

Debug builds are not just "slower tests": the runtime self-checks its invariants only there — every StateWord transition asserts its precondition, enqueue asserts the exact (gen, Queued) word, RawMutex enforces the never-two-cold-locks leaf rule with a per-thread held-count, live_actors checks for double-finalize underflow. A green release run with a red debug run means an invariant broke without (yet) corrupting behavior — treat it as a real failure.

The queue-variant matrix

The run queue is compile-time selected; the suite must pass under all three (features are additive, so drop the default first):

cargo test                                              # rq-mutex (default)
cargo test --no-default-features --features rq-mpmc
cargo test --no-default-features --features rq-striped

Loom (model checking)

RUSTFLAGS="--cfg loom" cargo test --lib --release

Exhaustively explores interleavings of the slot state machine (src/slot_state.rs: lost-wakeup, at-most-once-enqueue, the stale-pid ABA theorem, unpark-vs-claim) and the ring queues (src/run_queue.rs: exactly-once through lap wraparound, push/pop races). Models run the production transitions through src/sync_shim.rs — std atomics normally, loom::sync under --cfg loom. RawMutex is deliberately not modeled: futexes can't be, and it's the textbook Drepper mutex3 with stress and unwind-safety tests of its own.

Trace feature

cargo test --features smarm-trace exists mainly to catch bit-rot in the te!() call sites; run it after touching scheduler paths.

Before a runtime-core PR

The full matrix, in rough order of bug-finding power per minute:

  1. cargo test (debug, default variant)
  2. debug under rq-mpmc and rq-striped
  3. cargo test --release
  4. loom
  5. cargo build --features smarm-trace

Catalog

Low-level units (no scheduler)

file covers
context.rs init_actor_stack + the naked-asm context-switch shims, poked directly
stack.rs the mmap'd stack allocator
pid.rs pid packing/equality

Feature areas (run under a real runtime)

file covers
runtime.rs Config, Runtime::run, re-running a runtime, correctness under genuine parallelism
scheduler.rs spawn / join / panic delivery / yield_now / self_pid
channel.rs send/recv (recv parks, so these need the runtime)
selective_recv.rs recv_match / try_recv_match
mutex.rs the actor-blocking Mutex<T> (lock parks)
timer.rs sleep ordering — time-sensitive, generous tolerances by design
io.rs block_on_io: blocking closures on the pool while the actor parks
io_epoll.rs wait_readable / wait_writable + the read/write sugar
preempt.rs explicit preemption via smarm::check!()
cancel.rs cooperative cancellation (request_stop) — the keystone semantics
monitor.rs monitor delivers exactly one Down; demonitor
link.rs bidirectional links + trap_exit
supervisor.rs one-for-one supervision
gen_server.rs call/cast round-trips, lifecycle callbacks, server-down detection

Regression & stress

file covers
stress.rs lost wakeups, pid-table pressure, thundering herds, panic isolation under concurrency. Where the phase-2 RefCell-migration bug was caught.
poison_stop.rs request_stop racing an alloc-under-lock must not poison/abort. See its header for the full story.
many_timers_multi_thread.rs multi-thread sleep-timer lost-wakeup regression

Conventions

  • Each test owns its runtime. init(Config::exact(N)) + rt.run(...); never share a Runtime between tests. Oversubscription (exact(4) on one core) is deliberate — forced interleaving at yield points is how single-core CI finds races at all.
  • Regression tests must be validated against the bug. A regression test that passes with the bug reintroduced is documentation, not a test. Reintroduce the fix's inverse locally and watch it fail before trusting it (poison_stop.rs went through exactly this: its first version never fired the sentinel under a lock, and was rewritten until it SIGABRT'd pre-fix).
  • Stochastic tests get the odds stacked. Use Config::alloc_interval(1) to make every allocation an observation point, many actors, and both phases of any every-other-allocation cadence (see poison_stop::self_stop_during_spawn...).
  • Time-based assertions use ordering, not durations. Assert "didn't return instantly" / "A woke before B", with generous tolerances; CI machines are slow and noisy.
  • New invariants added to the runtime should come with the assert at the point of reliance (debug_assert on hot paths) and, where the invariant is a protocol, a loom model in the owning module — that combination is what made phases 2–5 land without a single post-merge race so far.