Root cause of soak20 signature 2 (refcount_test.exs 'watcher crash',
110235x fast {:error, :server_down} probes over the full await window):
by_name mapped name -> slot *index*, so a name whose holder died (no stop
path unregisters; prune is lazy) and whose slot was then re-tenanted read
as live-held: register failed NameTaken{holder: <unrelated tenant>} (which
the bridge macro's generated start() swallows -> start_server/1 reports :ok
for a server that never came up), while name resolution reached the
tenant's mailbox, missed on the message TypeId and failed fast WITHOUT
pruning — the wedge self-sustained for the tenant's lifetime. Name-
addressed send additionally judged liveness on the slot's *current*
mailbox pid, so a same-typed tenant would have received the message
(misdelivery) and a differently typed one a misleading NoChannel.
Fix: by_name: HashMap<&'static str, Pid> — every reader judges the
*stored* holder with the generation-checked live(), so a recycled slot's
tenant no longer impersonates a dead holder, and every touch (register /
whereis / resolve / send) prunes and heals a stale name. prune(index)
becomes prune_holder(pid): names bound to the holder go; the mailbox goes
only while still the holder's own (a tenant's replacement mailbox is left
untouched). Introspection matches names to mailboxes by full pid, so a
stale name never annotates a slot's new tenant.
Deterministic regression test added first and shown to fail pre-fix
(tests/stale_name_slot_reuse.rs: tiny slab forces re-tenanting; old slot
(1,0) died, tenant (1,1) took the index; register -> NameTaken pre-fix).
Post-fix it asserts the healed contract: whereis -> None (pruned), call ->
ServerDown, re-register -> Ok. Suite 33 ok-binaries, clippy gate clean.
In the wild the window opened at every splice_test teardown:
Splice.terminate -> exit_server('subtree') left the name bound; width 20
raised the re-tenant probability. Downstream (smarm_beam): install_child's
unregister-before-register workaround becomes dead code (removed there);
the #[smarm_server] macro's swallowed register error becomes truthful
idempotency (a NameTaken now really is a live holder).
Tests
Integration tests for the runtime. Each file owns one feature area or one
class of bug. Everything here runs under plain cargo test; the loom model
tests are the exception — they live in the library (src/slot_state.rs,
src/run_queue.rs), not in this directory, because loom must compile the
production code with shimmed atomics (see "Loom" below).
Running
cargo test # debug build — RUN THIS ONE: all invariant asserts live
cargo test --release # what users actually execute (LTO, no debug_asserts)
Debug builds are not just "slower tests": the runtime self-checks its
invariants only there — every StateWord transition asserts its
precondition, enqueue asserts the exact (gen, Queued) word, RawMutex
enforces the never-two-cold-locks leaf rule with a per-thread held-count,
live_actors checks for double-finalize underflow. A green release run with
a red debug run means an invariant broke without (yet) corrupting behavior —
treat it as a real failure.
The queue-variant matrix
The run queue is compile-time selected; the suite must pass under all three (features are additive, so drop the default first):
cargo test # rq-mutex (default)
cargo test --no-default-features --features rq-mpmc
cargo test --no-default-features --features rq-striped
Loom (model checking)
RUSTFLAGS="--cfg loom" cargo test --lib --release
Exhaustively explores interleavings of the slot state machine
(src/slot_state.rs: lost-wakeup, at-most-once-enqueue, the stale-pid ABA
theorem, unpark-vs-claim) and the ring queues (src/run_queue.rs:
exactly-once through lap wraparound, push/pop races). Models run the
production transitions through src/sync_shim.rs — std atomics normally,
loom::sync under --cfg loom. RawMutex is deliberately not modeled:
futexes can't be, and it's the textbook Drepper mutex3 with stress and
unwind-safety tests of its own.
Trace feature
cargo test --features smarm-trace exists mainly to catch bit-rot in the
te!() call sites; run it after touching scheduler paths.
Before a runtime-core PR
The full matrix, in rough order of bug-finding power per minute:
cargo test(debug, default variant)- debug under
rq-mpmcandrq-striped cargo test --release- loom
cargo build --features smarm-trace
Catalog
Low-level units (no scheduler)
| file | covers |
|---|---|
context.rs |
init_actor_stack + the naked-asm context-switch shims, poked directly |
stack.rs |
the mmap'd stack allocator |
pid.rs |
pid packing/equality |
Feature areas (run under a real runtime)
| file | covers |
|---|---|
runtime.rs |
Config, Runtime::run, re-running a runtime, correctness under genuine parallelism |
scheduler.rs |
spawn / join / panic delivery / yield_now / self_pid |
channel.rs |
send/recv (recv parks, so these need the runtime) |
selective_recv.rs |
recv_match / try_recv_match |
mutex.rs |
the actor-blocking Mutex<T> (lock parks) |
timer.rs |
sleep ordering — time-sensitive, generous tolerances by design |
io.rs |
block_on_io: blocking closures on the pool while the actor parks |
io_epoll.rs |
wait_readable / wait_writable + the read/write sugar |
preempt.rs |
explicit preemption via smarm::check!() |
cancel.rs |
cooperative cancellation (request_stop) — the keystone semantics |
monitor.rs |
monitor delivers exactly one Down; demonitor |
link.rs |
bidirectional links + trap_exit |
supervisor.rs |
one-for-one supervision |
gen_server.rs |
call/cast round-trips, lifecycle callbacks, server-down detection |
Regression & stress
| file | covers |
|---|---|
stress.rs |
lost wakeups, pid-table pressure, thundering herds, panic isolation under concurrency. Where the phase-2 RefCell-migration bug was caught. |
poison_stop.rs |
request_stop racing an alloc-under-lock must not poison/abort. See its header for the full story. |
many_timers_multi_thread.rs |
multi-thread sleep-timer lost-wakeup regression |
Conventions
- Each test owns its runtime.
init(Config::exact(N))+rt.run(...); never share aRuntimebetween tests. Oversubscription (exact(4)on one core) is deliberate — forced interleaving at yield points is how single-core CI finds races at all. - Regression tests must be validated against the bug. A regression test
that passes with the bug reintroduced is documentation, not a test.
Reintroduce the fix's inverse locally and watch it fail before trusting it
(
poison_stop.rswent through exactly this: its first version never fired the sentinel under a lock, and was rewritten until it SIGABRT'd pre-fix). - Stochastic tests get the odds stacked. Use
Config::alloc_interval(1)to make every allocation an observation point, many actors, and both phases of any every-other-allocation cadence (seepoison_stop::self_stop_during_spawn...). - Time-based assertions use ordering, not durations. Assert "didn't return instantly" / "A woke before B", with generous tolerances; CI machines are slow and noisy.
- New invariants added to the runtime should come with the assert at the point of reliance (debug_assert on hot paths) and, where the invariant is a protocol, a loom model in the owning module — that combination is what made phases 2–5 land without a single post-merge race so far.