18 Commits
Author SHA1 Message Date
Claude (sandbox) 301e3463e3 chore(release): v0.6.0 — RFC 019: actor stack reserve & shrink
Per-actor stack shapes on every spawn surface (SpawnOpts stack_reserve/
guard_size, Config defaults, pool rule: only default-shaped recycle);
sampled stack high-water + MADV_FREE shrink at actor-park (THRESHOLD
256 KiB, COOLDOWN 64 parks, redzone 1 page); pool-recycle MADV_DONTNEED
above the retained 64 KiB entry end; SIGSEGV overflow diagnostics
(two-tier: in-guard definitive / 1 MiB overshoot 'stepped over', prior
handler chained for foreign faults) with per-scheduler sigaltstack; and
the per-actor introspection surface (ActorInfo.stack: reserve, guard,
sampled depth, parks_since_shrink, shrinks).

Amendments ratified during implementation, for the RFC changelog:
- DEFAULT_STACK_GUARD 64 KiB -> 1 MiB, following the kernel's post-Stack-
  Clash stack_guard_gap convention; PROT_NONE width is VA-only and free.
- §7's motivating segfault was a cargo-vendored gz build, not SQLite as
  the RFC text says (cc-built C lacks -fstack-clash-protection; distro
  libraries have it — the risky class is vendored builds).
- §4 hibernate() deferred to the jar (bolt-on: force-flag on the §3
  shrink path, ~10 lines when wanted).

Gates (jobrunner box, 2026-08-08): reclaim gate PASS at c3 and again at
tip (3.0 MiB LazyFree -> kernel reclaim -> Rss to one live page ->
re-spike bit-identical, live data intact; MADV_PAGEOUT stands in for
memcg — cgroup2 is RO in the job container — driving the same reclaim
path). E1 interleaved A/B vs v0.5.0: every ka cell (the E1 subject)
within +0.3..+2.9% at tip; close-mode control cells within noise except
t8-c4 close, which is bistable (~40-44k vs ~46-49k modes for BOTH
variants, base self-disagrees by 11% across rounds); 6 rounds across two
runs are inconclusive there and a 10-round focused run is noted in the
handoff as deferred follow-up, accepted for this release.

No breaking API changes since v0.5.0: SpawnOpts fields and ActorInfo
gained members (exhaustive-construction downstream will need the new
ActorInfo.stack field; urus does not construct it).
2026-08-08 19:44:55 +00:00
Claude (sandbox) 410ba33d82 feat(introspect,runtime): per-actor stack surface on ActorInfo (RFC 019 §8)
- introspect::StackInfo { reserve, guard, depth_high_water,
  parks_since_shrink, shrinks } as ActorInfo.stack; re-exported at crate
  root beside ActorInfo.
- All reads lock-free: geometry from the c6 diag slot atomics, depth =
  top - hwm (the §2 sampled high-water; doc spells out sampled-not-exact
  and that 0 means never-descheduled-at-depth), counters straight off the
  §3 atomics. Coherence for the incarnation rides read_slot's existing
  generation check, same as overruns/messages_received.
- Slot::stack_introspect(): one pub(crate) tuple accessor beside the other
  counter accessors.
- Exact RSS deliberately absent per RFC (mincore = debug tooling only,
  never a runtime path); stack_shape(pid) untouched (cold-lock exact
  variant from c2).
- tests/introspect.rs: defaults surface (64 KiB reserve / 1 MiB guard /
  sampled ~32 KiB depth / gate park counted / zero shrinks) + live shrink
  counters (spike visible pre-shrink; shrinks>=1, cooldown counter reset,
  hwm reset after crossing COOLDOWN) read mid-run -- post-join the slot
  reclaim correctly hides the incarnation, which the first draft of the
  test learned the hard way.

FLAGGED (Claude-solo calls):
- Nested StackInfo struct over five flat ActorInfo fields (grain break;
  the five fields are one concern and ActorInfo is already 12 fields).
- Field names reserve/guard/shrinks (RFC says stack_reserve/stack_guard/
  shrink count; the stack_ prefix is redundant inside StackInfo).
2026-08-08 19:12:46 +00:00
Claude (sandbox) 5fd8aecf55 feat(signal,runtime,stack): SIGSEGV overflow diagnostics + 1 MiB guard default (RFC 019 §7)
- src/signal.rs: process-global SA_SIGINFO|SA_ONSTACK handler installed once
  at runtime::init (before any scheduler thread -> unracing PRIOR save);
  per-scheduler-thread 64 KiB sigaltstack registered at schedule_loop entry
  (a guard hit leaves no stack to handle on). Async-signal-safe throughout:
  classification is plain loads (const-init TLS Cell + slot atomics), print
  is fixed-buffer itoa + one write(2), death is SIG_DFL + refault at the
  same instruction (core-dumpable, correct wait status).
- Two-tier classification (agreed): in-guard = definitive; OVERSHOOT window
  below the guard = 'unprobed (FFI?) frame stepped over it' probable
  attribution -- the RFC's motivating incident (cargo-vendored gz, not
  SQLite as the RFC text says) faults there under a small guard. Pure
  classify() fn, 5 adversarial units incl. saturation at low addresses.
- DEFAULT_STACK_GUARD 64 KiB -> 1 MiB (agreed): kernel stack_guard_gap
  anchor post-Stack-Clash; PROT_NONE is VA-only (no RSS, no page tables,
  no overcommit charge) so width is free at any actor count.
- Unclassified faults reinstate the PRIOR sigaction and refault (agreed):
  std's own OS-thread overflow diagnostics survive our presence.
- Slot: diag_{stack_top,stack_reserve,stack_guard,pid} atomics written in
  install_actor pre-publish; readable without the cold lock (Stack lives
  under it); only consulted while CURRENT_SLOT points at the slot, so
  never stale where read. preempt::current_slot_ptr ungated from
  smarm-causal (now also the classifier's anchor).
- build.rs + cc (agreed Q3): canary/canary.c, 96 KiB local touched low-end
  first, -fno-stack-clash-protection pinned so hardened toolchains don't
  probe the canary into uselessness.
- tests/stack_diag.rs: subprocess x4 -- Rust recursion tier-1; FFI canary
  tier-1 at defaults (1 MiB guard catches the jump); tier-2 at guard=4 KiB
  ('stepped over', reproduces the incident); clean at reserve=256 KiB
  (the §1 knob is the fix, same frame).

FLAGGED (Claude-solo calls):
- OVERSHOOT_SLOP = 1 MiB (matches guard default/kernel gap; beyond it
  attribution would be dishonest).
- Altstack 64 KiB, mmap'd once per OS thread, never freed (bounded by
  thread count; reused across run()s via TLS flag).
- Foreign-fault reinstate permanently deregisters our handler; accepted --
  the process is dying either way.
- Diag geometry as 4 slot atomics (install-time cost only) over a per-switch
  TLS snapshot (hot-path stores).
2026-08-08 18:58:30 +00:00
Claude (sandbox) 7d8b9e0310 feat(stack,runtime): pool-recycle DONTNEED above the retained entry end (RFC 019 §6)
- stack::retain_range: pure checked span fn (retain page-up = zap less;
  None when retain covers the reserve, so the 64 KiB default config never
  pays a syscall) + 6 adversarial units mirroring shrink_range's.
- Stack::recycle_zap: advisory MADV_DONTNEED of [usable_base, top-RETAIN);
  stack is unowned at the call site, synchronous eager zap races nothing.
- recycle_stack: zap OFF-LOCK before pool admission (acquire_stack's
  no-syscall-under-the-pool-lock invariant); rare cap-overflow pays a
  wasted zap ahead of munmap, accepted over a second lock round-trip.
- pub const RECYCLE_RETAIN = 64 KiB beside the shrink knobs, ratified-as-
  constant rationale in doc.
- tests/stack_recycle.rs: mincore-based exact-zero-resident assert over
  the zap span. smaps was tried first and over-counts: a neighboring rw
  anon VMA can merge flush against the stack top (observed once under the
  full-suite run); the PROT_NONE guard pins the usable base exactly.

FLAGGED (Claude-solo calls):
- RFC §6 'above the bottom RETAIN' is direction-ambiguous in address
  terms; implemented as retain the ENTRY end (highest addresses, the
  pages the next actor faults first), zap the cold deep span below.
- Const named RECYCLE_RETAIN (RFC says RETAIN) to sit beside SHRINK_*.
2026-08-08 16:13:53 +00:00
Claude (sandbox) 8225716b11 feat(runtime,stack): sampled stack high-water + MADV_FREE shrink at actor-park (RFC 019 §§2–3)
hwm: AtomicUsize lands beside sp on the slot: the single context-save
site min-updates it (one branch + at most one Relaxed store into the
line the sp store just dirtied), install resets it to the fresh top.
Advisory by construction — correctness never depends on it. The mod-doc
ordering chain gains a line: hwm piggybacks the existing
Relaxed-store-before-Release pattern and adds no edges.

Shrink hook in the YieldIntent::Park arm only, before the park_return
Release transition — the owned window (obligation 1's assert-comment at
the site): after the sp store, before Parked is published, scheduler on
its own stack, actor saved and unstealable. It runs on both arms of the
park_return race (a consumed unpark flag means one wasted-but-harmless
madvise). The preempt/yield path deliberately never checks: §4's
bounded, self-healing leak under saturation, when syscalls are least
affordable.

SHRINK_THRESHOLD = 256 KiB and SHRINK_COOLDOWN = 64 parks are pub
constants with the ratified doc rationale, not Config fields. The freed
span is shrink_range(hwm, sp, page): whole pages of [hwm, sp − 1-page
redzone), rounded inward, checked arithmetic — adversarial inputs
collapse to None (obligation 2). MADV_FREE marks lazily; the kernel's
reclaim-under-pressure IS the hysteresis, cancel-on-write is the safety
net. parks_since_shrink + shrink_count ride the slot for the cooldown
and the future introspect surface.

Tests: 7 adversarial shrink_range units (inverted/empty spans, redzone
underflow, unaligned ends, sp-crossing sweep); integration — 8 MiB
reserve, ~3 MiB spike sampled via yield-at-depth, parks gated on
introspected Parked state past the cooldown, then ≥ 2 MiB LazyFree
asserted inside the stack's smaps range with live data intact; and the
inverse guard — a shallow never-spiking actor ends at exactly 0
LazyFree (also proves the parser isn't vacuously zero via the first
test).
2026-08-08 14:30:32 +00:00
Claude (sandbox) 3cb64eefc2 feat(scheduler,gen_server,gen_statem,introspect): SpawnOpts — per-actor stack shape on every spawn surface (RFC 019 §1)
SpawnOpts { stack_reserve, guard_size } with Option<usize> fields, None
resolving to the Config defaults at spawn time — a deliberate deviation
from the RFC's plain-usize struct so struct-update syntax works without
a runtime handle in scope. Threaded across the five surfaces:
spawn_with, spawn_under_with, spawn_addr_with,
GenServerBuilder::stack_opts (mirrored on NamedGenServerBuilder), and
gen_statem::spawn_with (gen_statem has no builder, so the opts ride a
_with variant — Claude-solo surface call, flagged for review). Existing
spawns forward defaults; no call-site churn.

introspect::stack_shape(pid) pulled forward (agreed) as the first slice
of the RFC 019 introspection surface, giving tests an observable.

Tests (tests/spawn_opts.rs): override/partial-override/rounding on each
surface; obligation 4 from the outside — a dead custom stack is never
handed to the next default spawn (LIFO pool would expose it), and the
reverse (default stacks ARE recycled); 8 MiB reserve behaviorally
permits ~1 MiB recursion. Also: silence unused-Result in the c1
runtime test (join now unwrapped).
2026-08-08 14:22:38 +00:00
Claude (sandbox) 0fe052bc7e feat(stack,runtime): per-shape actor stacks — Stack::new(reserve, guard), Config knobs, pool rule (RFC 019 §1)
Stack takes an explicit (reserve, guard) shape, both page-rounded and
stored; usable_base derives from the stored guard. Guard default raised
4 KiB -> 64 KiB (DEFAULT_STACK_GUARD): probestack makes one page enough
for Rust frames, but an unprobed C frame can leap a page in one sub rsp
— the motivating SQLite segfault. Reserve default stays 64 KiB
(DEFAULT_STACK_RESERVE); ACTOR_STACK_SIZE retired.

Config::{stack_reserve, stack_guard} thread the runtime defaults into
RuntimeInner pre-rounded. All acquisition/recycling now goes through
acquire_stack/recycle_stack carrying the pool rule: only default-shaped
stacks are pooled (pooled ⇒ default-shaped by induction); custom shapes
mmap fresh and munmap at death. Pool lock still dropped before any mmap.

No public spawn API change (SpawnOpts is the next commit).

Tests: shape rounding + accessors, wide-guard faults at both ends
(subprocess), Config::stack_reserve permits >64 KiB recursion that
previously could only segfault.
2026-08-08 14:18:27 +00:00
smarm a03a7ca01e chore(release): v0.5.0
Breaking API rename since v0.4.0: gen_server's ServerRef/ServerBuilder/
ServerCtx -> GenServerRef/GenServerBuilder/GenServerCtx, Watcher<G> is
now generic over its GenServer, and GenServer gained a required
associated Timer type for timer-fire payloads (arm_after/handle_timer).
Downstream consumers (urus) have been ported.
2026-08-08 11:44:35 +02:00
smarm d4839f1d81 feat(runtime,io): driver-enqueues + park/wake idle path — retire the wake pipe
The swap (RFC 018). Schedulers no longer sleep on a shared level-triggered
wake pipe — the herd source that made the default 8-thread config 7x
slower than 2 threads (E1). They park on per-thread futex parkers via the
coordination layer; IO backends become producers behind a two-call
contract (make runnable, then the enqueue tail wakes exactly one parked
scheduler).

Deleted: the drain lock and the one-winner phase-1 drain; the shared
completions VecDeque; the wake pipe fds, poll_wake, drain_wake_pipe,
wake_scheduler, the FdReady/Blocking Completion enum; the 100us idle nap;
the per-pop io.lock liveness read; io.rs's as_millis timeout truncation.

Added:
- enqueue wake tail (fixes the silent enqueue): wake_one_if_idle, a fence
  + one Relaxed mask load when everyone is busy — the pure-compute hot
  path pays almost nothing.
- driver-enqueues: the pool thread stashes its result in the slot,
  decrements io_outstanding, unparks; the epoll thread removes+DELs the
  waiter under the waiters lock and unparks. Both reach the runtime via a
  Weak (no Arc cycle). The waiters map moves behind its own Arc<Mutex> so
  the epoll thread never takes the runtime io lock (teardown holds it
  while joining that thread).
- io_outstanding / io_fd_waiters atomics: the termination verdict reads
  two atomics instead of taking io.lock on every pop.
- timekeeper idle path: at most one parked scheduler holds the timer
  deadline (an expiry wakes one, not a herd); everyone else parks
  indefinitely and is woken by the enqueue tail.
- busy-path timer due-check (ratified design point (a)): under saturation
  nobody parks and no timekeeper exists, yet due timers must still fire —
  one Relaxed load of the earliest-deadline snapshot per loop, clock read
  only when a timer is armed. Maintained under the timers mutex.
- chain rule: a scheduler that pops with more work queued and a sibling
  parked wakes one, so surplus runs in parallel rather than behind it.

tests/park_wake.rs pins the two new observable properties: timers fire
under full scheduler saturation, and sub-ms sleeps are prompt (the
as_millis truncation regression). Full suite + all loom models green;
clippy --lib clean.
2026-07-24 09:12:12 +02:00
smarm 2854b560d6 feat(park): fenced producer fast path + earliest-deadline snapshot
Two integration-driven amendments ahead of the runtime swap:

wake_one_if_idle() realizes RFC 018's "empty-mask fast path is one
relaxed load" soundly: a bare relaxed load is a lost-wake in the Dekker
shape for the lock-free ring queues, so the producer publishes work,
fences (SeqCst), then reads the mask Relaxed — paired with a matching
fence between the consumer's bit-publish and its re-check in park().
The pure-compute hot path (mask 0) never takes the shared mask line
exclusive; the RMW read stays on the rare chain-rule path only. Loom
models 1/2 now drive the fenced pattern end to end.

next_deadline is the earliest KNOWN timer deadline, independent of
whether anyone is parked — which tk_armed cannot give: under saturation
nobody parks, nobody arms, yet due timers must still fire (ratified
design point (a): the busy-path due-check). Maintained under the timers
mutex (note_deadline on insert — which also carries the timekeeper
re-arm wake — refresh_deadline after pop/clear); read lock-free.
deadline_due() costs one Relaxed load and a branch when no timer exists;
the clock is read only when one does.
2026-07-24 09:12:12 +02:00
smarm 7b026cfe56 feat(park): scheduler coordination layer — parkers, idle mask, wake protocol (RFC 018)
Schedulers get an IO-agnostic sleep/wake primitive of their own: one
futex Parker per scheduler thread (permit semantics, std::thread::park
shaped — closes the check-then-park race), an AtomicU64 idle mask with a
set-bit → re-check → wait park protocol, wake_one (highest-bit LIFO,
CAS-clear before unpark: exactly one wakeup per call by construction),
wake_all for the terminal path, and the timekeeper role — at most one
parked scheduler holds the timer deadline, with an atomic armed-deadline
snapshot for the busy-path due-check and an insert-side re-arm wake.

Deadlines travel as nanosecond timespecs end to end; the wake pipe's
as_millis truncation is unrepresentable here. The Dekker publish/re-check
shape is resolved by the same-location-RMW handshake (AcqRel), not SeqCst
loads; loom verifies exactly this in four models (no-lost-wake, chain
propagation, timekeeper handoff, termination), run with
LOOM_MAX_PREEMPTIONS=3 — unbounded exploration is impractical for the
looped models. Loom/non-Linux builds park on a Mutex+Condvar via
sync_shim.

Standalone until the runtime swap (next commit): nothing outside tests
constructs a Coordinator yet, hence the temporary dead_code allow in
lib.rs.
2026-07-24 09:12:12 +02:00
smarm 006a3283e7 chore(hooks): clippy gate falls back to a nix-shell toolchain
Desktop migration: the home-manager rust here ships without the clippy
component. Prefer an installed cargo-clippy; otherwise run clippy from an
ephemeral nix-shell with a separate target dir (mixed-compiler artifacts
are an E0514 hard error). MSRV keeps the shell's older toolchain a
legitimate gate.
2026-07-24 09:12:12 +02:00
Markk116 8c764e9169 docs(monitor): user-facing rewrite of process monitors
Lead with the user's problem (learn when another actor dies without
it knowing you're watching), explain one-directional/one-shot
semantics and contrast briefly with link without assuming link.rs has
been read. Add a compiling doctest. Drop em-dashes. Correctness facts
about registration/death races and demonitor-after-fire safety kept,
reworded in plain terms and separated from the public item docs.
2026-07-24 08:44:56 +02:00
Markk116 41b9d6d056 docs(introspect): user-facing rewrite of runtime introspection
Was the worst offender for external-context references (RFC 016
Chunk 1/4, DECISION D1/D2, RFC 003/011), all removed. Lead with the
practical use cases (debugging, health checks, test assertions,
dashboards) for snapshot()/actor_info()/tree(), and explain the
per-actor-reads-not-a-world-freeze consistency model in plain terms
instead of citing a decision log. Add a compiling doctest.
2026-07-24 08:44:56 +02:00
Markk116 dd845f22fe docs(mutex): user-facing rewrite of the actor-blocking mutex
Every public item was previously undocumented. Lead with why Mutex<T>
exists (a channel/gen_server is overkill for plain shared state) and
how it differs from std::sync::Mutex (parks the actor not the OS
thread, every lock is timeout-bounded by default). Add a compiling
doctest. Document new/lock/lock_timeout/try_lock/set_default_timeout/
MutexGuard/LockTimeout/DEFAULT_TIMEOUT. Drop em-dashes; keep wake
protocol mechanics as contributor-facing comments on private internals.
2026-07-24 08:44:56 +02:00
Markk116 36a0a9832d docs(registry): user-facing rewrite of the name registry
Replace the 'what changed' diff-against-a-prior-design framing with a
plain explanation of what the registry is for (naming an actor so
others can find and message it by name) and a compiling doctest
(register/whereis/send/unregister). Cut all RFC/decision-number/bug-id
references and em-dashes; move type-erasure and locking-discipline
detail into an Implementation notes section for contributors.
2026-07-24 08:40:26 +02:00
Markk116 feda6517e5 docs(channel): user-facing rewrite of the MPSC channel primitive
Lead with what a channel is and how to use it (compiling doctest for
channel()/send/recv/close), before any internal rationale. Document
every previously-undocumented public item (channel(), Sender, Receiver,
SendError, RecvError). Move the RawMutex-vs-std::sync::Mutex rationale
and lock-class discipline into an Implementation notes section. Drop
em-dashes throughout.
2026-07-24 08:40:26 +02:00
Markk116 8625ae4c35 docs(scheduler): user-facing rewrite of the actor/spawn/run entry point
Lead with what an actor is and how to start one with run()/spawn(),
following gen_server.rs's example-first style. Add a compiling module
doctest. Drop RFC references and em-dashes; keep internal mechanics
(preemption gating, thread-local borrow rules) as plain contributor
comments rather than public-facing doc prose.
2026-07-24 08:40:26 +02:00
31 changed files with 4694 additions and 1213 deletions
+17 -1
View File
@@ -2,7 +2,23 @@
# smarm pre-commit gate: clippy the library (src/) with warnings as errors.
# unwrap_used / expect_used are denied (Cargo.toml [lints.clippy]): library
# code must not hide a panic behind unwrap/expect. Tests/examples are not gated.
#
# Toolchain resolution: prefer an installed cargo-clippy; on machines whose
# rust comes without the clippy component (e.g. NixOS home-manager), fall
# back to an ephemeral nix-shell toolchain. The fallback uses its own target
# dir (target/clippy) because the shell's rustc version may differ from the
# default toolchain's — mixed-compiler artifacts in one target dir are an
# E0514 hard error. MSRV (Cargo.toml rust-version) keeps the older shell
# toolchain a legitimate gate.
set -eu
[ -f "$HOME/.cargo/env" ] && . "$HOME/.cargo/env"
cd "$(git rev-parse --show-toplevel)"
cargo clippy --lib -- -D warnings
if cargo clippy --version >/dev/null 2>&1; then
cargo clippy --lib -- -D warnings
elif command -v nix-shell >/dev/null 2>&1; then
nix-shell -p clippy -p cargo -p rustc \
--run 'CARGO_TARGET_DIR=target/clippy cargo clippy --lib -- -D warnings'
else
echo "pre-commit: cargo clippy unavailable and no nix-shell fallback" >&2
exit 1
fi
+4 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "smarm"
version = "0.4.0"
version = "0.6.0"
edition = "2021"
rust-version = "1.95"
@@ -39,6 +39,9 @@ rq-mutex = []
rq-mpmc = []
rq-striped = []
[build-dependencies]
cc = "1"
[dependencies]
libc = "0.2"
+11
View File
@@ -0,0 +1,11 @@
fn main() {
// RFC 019 §7 test canary (agreed Q3): compiled without stack-clash
// protection so its 96 KiB local is a genuine one-displacement guard
// jumper; distro-hardened compilers would otherwise probe it page-wise
// and defeat the test's purpose.
cc::Build::new()
.file("canary/canary.c")
.flag_if_supported("-fno-stack-clash-protection")
.compile("smarm_canary");
println!("cargo:rerun-if-changed=canary/canary.c");
}
+14
View File
@@ -0,0 +1,14 @@
/* RFC 019 §7 FFI canary: an honest unprobed C frame with a 96 KiB local,
* touched from its LOW end first — the exact "one sub rsp steps over a small
* guard" pattern the RFC's motivating incident hit (a cargo-vendored gz
* build; cc-invoked builds do not enable -fstack-clash-protection, and this
* file pins that off explicitly so the canary stays a canary even on
* hardened-default toolchains). */
void smarm_canary_burn(void) {
volatile char buf[96 * 1024];
buf[0] = 1; /* deepest address first */
for (unsigned i = 0; i < sizeof buf; i += 4096) {
buf[i] = (char)i;
}
buf[sizeof buf - 1] = 1;
}
+1 -1
View File
@@ -75,7 +75,7 @@ genuine advantage over tokio's task abort model.
### Spawn-heavy workloads (19–70×)
Every smarm actor `mmap`s a 64 KiB stack with a guard page. This is
Every smarm actor `mmap`s a 64 KiB stack reserve with a 64 KiB PROT_NONE guard below (both per-actor configurable since RFC 019; the reserve is demand-paged). This is
a syscall. Tokio tasks are heap-allocated state machines — no stack,
no syscall, ~100 bytes each. For workloads that spawn thousands of
short-lived actors per second, this is a structural disadvantage.
+275 -205
View File
@@ -1,38 +1,102 @@
//! Unbounded MPSC channels.
//! Unbounded multi-producer, single-consumer channels: how actors talk to
//! each other.
//!
//! Inner state is `Arc<RawMutex<Inner<T>>>` so channels can be sent across OS
//! threads (required for the multi-scheduler runtime where a sender and
//! receiver may run on different scheduler threads simultaneously).
//! A channel is a queue with a typed [`Sender`] on one end and a typed
//! [`Receiver`] on the other. Any number of actors can hold a clone of the
//! `Sender` and push messages onto the same queue; exactly one [`Receiver`]
//! reads them back out, in the order they arrived. This is the basic wiring
//! smarm's other actor primitives (`gen_server`, `pg`, the registry) are all
//! built out of, and it is directly usable on its own for a worker that just
//! needs an inbox.
//!
//! ## Why `RawMutex` (Channel class), not `std::sync::Mutex`
//! ## A first channel
//!
//! An actor holding a guard with preemption *enabled* can be timesliced
//! inside the critical section and resume on a different OS thread — the
//! pthread mutex would then be released from a thread that didn't lock it,
//! which is UB (Linux futexes happen to tolerate it, but it's not
//! guaranteed). `RawMutex` disables preemption for the guard's span and is
//! cross-thread-release sound by construction, closing the hole. It also
//! cannot poison. Channel locks form their own [`LockClass::Channel`]
//! (raw_mutex.rs): they may be taken under a cold (Leaf) lock — finalize and
//! `monitor()` clone senders that live in slots — but nothing may be locked
//! under them, which the debug build enforces. `recv_match` runs its user
//! predicate under this lock: keep it cheap, pure, and channel-free.
//! ```
//! use smarm::{channel, run, spawn};
//!
//! Semantics:
//! - Senders are clonable; the last sender drop closes the channel.
//! - `Receiver::recv` on an empty open channel parks the receiver.
//! - `Receiver::recv` on an empty closed channel returns `Err(RecvError)`.
//! - `Sender::send` on an open channel always succeeds.
//! - `Sender::send` on a closed channel (receiver dropped) returns
//! `Err(SendError(value))`.
//! - When a send pushes to a previously empty queue and a receiver is
//! parked, the receiver is unparked.
//! run(|| {
//! let (tx, rx) = channel::<u64>();
//!
//! let worker = spawn(move || {
//! // Blocks until a message arrives.
//! let n = rx.recv().unwrap();
//! assert_eq!(n, 42);
//!
//! // Once every Sender is dropped, recv() reports the channel closed
//! // instead of blocking forever.
//! assert!(rx.recv().is_err());
//! });
//!
//! tx.send(42).unwrap();
//! drop(tx); // last sender gone: the channel is now closed
//! worker.join().unwrap();
//! });
//! ```
//!
//! ## Sending
//!
//! [`Sender`] is cheaply clonable: hand a clone to every actor that needs to
//! push messages into this queue. The channel stays open as long as at least
//! one clone exists; [`Sender::send`] never blocks and always succeeds while
//! the channel is open, since the queue is unbounded. Once the [`Receiver`]
//! has been dropped, `send` returns the message back to you in
//! [`SendError`] instead of delivering it.
//!
//! ## Receiving
//!
//! There is exactly one [`Receiver`] per channel (it is not clonable).
//! [`Receiver::recv`] returns the next message in arrival order, parking the
//! calling actor if the queue is currently empty. Once every `Sender` has
//! been dropped and the queue has been drained, `recv` stops parking and
//! returns [`RecvError`] instead, so a receiver never blocks forever waiting
//! on senders that are never coming back.
//!
//! Beyond plain `recv`, three variants cover the common needs:
//!
//! - [`Receiver::try_recv`]: never parks: reports an empty-but-open channel
//! as `Ok(None)` instead of waiting.
//! - [`Receiver::recv_timeout`]: parks, but gives up and returns
//! [`RecvTimeoutError::Timeout`] if no message arrives before a deadline.
//! - [`Receiver::recv_match`] / [`Receiver::try_recv_match`]: selective
//! receive. Instead of taking whatever is at the front of the queue, pick
//! out the first message matching a predicate, leaving the rest queued in
//! order. Handy for an actor that wants to prioritise one kind of message
//! over others already waiting.
//!
//! ## Waiting on several channels: `select`
//!
//! [`select`] parks an actor across several receivers at once and reports
//! the index of the first one that is ready (has a message queued, or has
//! been closed). [`select_timeout`] adds a deadline, the way `recv_timeout`
//! does for a single channel. See their docs for the full contract,
//! including the priority-order and no-fairness guarantee.
//!
//! ## Implementation notes
//!
//! The queue and its bookkeeping live behind `Arc<RawMutex<Inner<T>>>`
//! rather than a `std::sync::Mutex`, so that a channel can be freely shared
//! and sent across the OS threads backing the multi-scheduler runtime.
//! `RawMutex` matters here for a subtler reason too: an ordinary pthread
//! mutex can be released from a different OS thread than the one that took
//! it (smarm's preemption can migrate a timesliced actor between scheduler
//! threads mid-critical-section), and doing that to a `std::sync::Mutex` is
//! undefined behavior. `RawMutex` disables preemption for the guard's short
//! lifetime instead, so the release always happens on the thread that
//! acquired it, and it has no poisoning to worry about besides. Channel
//! locks are cheap and are never held across another lock acquisition or a
//! blocking call; the predicate passed to `recv_match` runs under this lock,
//! which is why it needs to stay cheap, pure, and must not call back into
//! the same channel.
use crate::pid::Pid;
use crate::raw_mutex::RawMutex;
use std::collections::VecDeque;
use std::sync::Arc;
/// Create a new channel and return its `(Sender, Receiver)` halves.
///
/// The channel is unbounded (no capacity limit) and open until every
/// `Sender` has been dropped.
pub fn channel<T>() -> (Sender<T>, Receiver<T>) {
let inner = Arc::new(RawMutex::new_channel(Inner {
queue: VecDeque::new(),
@@ -45,29 +109,40 @@ pub fn channel<T>() -> (Sender<T>, Receiver<T>) {
struct Inner<T> {
queue: VecDeque<T>,
/// The parked receiver's `(pid, park-epoch)`. The epoch is the slot
/// word's runtime-wide wait identity (see slot_state.rs): wakers call
/// `unpark_at(pid, epoch)`, so an entry left over from an already-woken
/// wait — a `select` loser arm, a satisfied `recv_timeout`'s timer — is
/// inert: the wake fails the word's epoch CAS and no-ops. This replaces
/// the old per-channel `cur_wait`/`next_wait_seq`/`timed_out` trio: wait
/// identity now exists exactly once, in the slot word.
/// The parked receiver's `(pid, park-epoch)`, if one is currently
/// waiting. The epoch identifies exactly which wait this is, so a waker
/// left over from a wait that already ended (a losing `select` arm, a
/// `recv_timeout` whose timer fired after it was already satisfied) is
/// inert and does nothing when it fires.
parked_receiver: Option<(Pid, u32)>,
senders: usize,
receiver_alive: bool,
}
/// The sending half of a channel, created by [`channel`]. Clonable: every
/// clone pushes onto the same queue, and the channel stays open as long as
/// any clone is alive. Dropping the last `Sender` closes the channel, which
/// wakes a parked [`Receiver`] so it can observe the closure.
pub struct Sender<T> {
inner: Arc<RawMutex<Inner<T>>>,
}
/// The receiving half of a channel, created by [`channel`]. Not clonable:
/// a channel has exactly one receiver. Reads messages in the order they
/// were sent, via [`recv`](Receiver::recv) and its variants.
pub struct Receiver<T> {
inner: Arc<RawMutex<Inner<T>>>,
}
/// Returned by [`Sender::send`] when the channel's [`Receiver`] has already
/// been dropped. Carries the message back so it is never silently lost;
/// recover it with `.0` or by matching.
#[derive(Debug, PartialEq, Eq)]
pub struct SendError<T>(pub T);
/// Returned by [`Receiver::recv`] (and the other receive methods, in their
/// own error types) when the channel is closed: every `Sender` has been
/// dropped and no message is left queued.
#[derive(Debug, PartialEq, Eq, Clone, Copy)]
pub struct RecvError;
@@ -84,8 +159,8 @@ impl std::error::Error for RecvError {}
pub enum RecvTimeoutError {
/// The deadline passed with no message available.
Timeout,
/// All senders dropped with no message available — the bounded analogue
/// of [`RecvError`].
/// Every sender was dropped with no message available. The
/// timeout-aware counterpart of plain [`RecvError`].
Disconnected,
}
@@ -115,8 +190,8 @@ impl<T> Drop for Sender<T> {
// Wake the parked receiver on the last sender drop regardless of
// whether the queue is empty. A plain `recv` only ever parks on an
// empty queue (so this is unchanged for it), but a selective
// `recv_match` may be parked on a *non-empty* queue holding only
// non-matching messages — it must wake to observe closure and
// `recv_match` may be parked on a non-empty queue holding only
// non-matching messages. It must wake to observe closure and
// return Err rather than sleep forever.
if g.senders == 0 {
g.parked_receiver.take()
@@ -133,14 +208,15 @@ impl<T> Drop for Sender<T> {
impl<T> Drop for Receiver<T> {
fn drop(&mut self) {
// The only consumer is gone: queued messages can never be delivered.
// Drop them now instead of stranding them until the last Sender goes
// away (a registry entry under lazy prune can keep a Sender — and thus
// the Arc<Inner> — alive long after the server exits). Dropping a queued
// Envelope::Call drops its reply_tx, waking any caller parked in `call`
// with ServerDown, so the documented guarantee holds on *every* teardown
// path, not only the all-senders-drop one. Drain under the lock, then
// run item destructors after releasing it (a reply_tx drop reaches into
// a *different* channel's lock + the scheduler, so it must not nest).
// Drop them now instead of leaving them queued until the last Sender
// happens to go away, which can be long after this receiver's owner
// has exited if some other part of the runtime is still holding a
// clone of the Sender. Draining runs each queued message's own drop
// glue, which matters for a gen_server call: dropping a queued call
// envelope drops its reply channel too, which wakes the caller with
// an error instead of leaving it parked forever. Drain under the
// lock, then run the drops after releasing it, since a message's
// drop glue may itself touch a different channel or the scheduler.
let drained = {
let mut g = self.inner.lock();
g.receiver_alive = false;
@@ -151,14 +227,17 @@ impl<T> Drop for Receiver<T> {
}
impl<T> Sender<T> {
/// Number of messages currently queued behind this channel. Introspection
/// only (RFC 016 mailbox depth); takes the channel lock, so callers reach
/// it under the registry Leaf (Leaf → Channel) via the erased probe in
/// `registry.rs`, never on a hot path.
/// Number of messages currently queued and not yet received. For
/// introspection and monitoring; takes the channel's internal lock, so
/// avoid calling it from a hot path.
pub(crate) fn queued_len(&self) -> usize {
self.inner.lock().queue.len()
}
/// Push `value` onto the channel. Succeeds unconditionally as long as
/// the [`Receiver`] is still alive: the queue has no capacity limit, so
/// this never blocks and never fails except when the channel is closed,
/// in which case `value` comes back in [`SendError`].
pub fn send(&self, value: T) -> Result<(), SendError<T>> {
let unpark = {
let mut g = self.inner.lock();
@@ -179,6 +258,10 @@ impl<T> Sender<T> {
}
impl<T> Receiver<T> {
/// Block until a message is available and return it. Messages come back
/// in the order they were sent. If the queue is empty and every
/// [`Sender`] has already been dropped, returns [`RecvError`] instead of
/// blocking forever.
pub fn recv(&self) -> Result<T, RecvError> {
loop {
{
@@ -198,15 +281,15 @@ impl<T> Receiver<T> {
g.parked_receiver.is_none_or(|(p, _)| p == me),
"channel has more than one receiver"
);
// begin_wait is lock-free — legal under the Channel lock;
// begin_wait is lock-free, so it's legal under the Channel lock;
// registering in the same critical section makes the epoch
// atomic with the senders' view of the registration.
g.parked_receiver = Some((me, crate::scheduler::begin_wait()));
crate::te!(crate::trace::Event::RecvPark(me));
}
// Release the lock before parking — the unparker will need it.
// Release the lock before parking: the unparker will need it.
crate::scheduler::park_current();
// Woken up — record it before looping to check the queue.
// Woken up. Record it before looping to check the queue.
crate::te!(crate::trace::Event::RecvWake(match crate::actor::current_pid() {
Some(p) => p,
None => panic!("smarm: RecvWake outside an actor (core corrupt)"),
@@ -214,29 +297,18 @@ impl<T> Receiver<T> {
}
}
/// Bounded receive: like [`recv`](Self::recv), but gives up once
/// `timeout` has elapsed, returning [`RecvTimeoutError::Timeout`].
/// Like [`recv`](Self::recv), but gives up and returns
/// [`RecvTimeoutError::Timeout`] if no message has arrived by the time
/// `timeout` elapses.
///
/// Built on the same timer machinery as `Mutex::lock_timeout`: the wait
/// registers a `WaitTimeout` entry stamped with the wait's park-epoch;
/// on expiry the channel (as the
/// [`TimerTarget`](crate::timer::TimerTarget)) checks whether *this*
/// wait is still parked and, only then, cancels it. A wake that races
/// the deadline resolves message-first: if a message is available when
/// the receiver runs, it is delivered even if the timer had already
/// fired. A satisfied or abandoned wait leaves its timer entry to expire
/// as a no-op (registration gone; epoch consumed), per the
/// no-cancellation convention in `timer.rs`.
/// If a message arrives at essentially the same moment the deadline
/// passes, the message wins: you get `Ok` rather than `Timeout`. If
/// every sender is dropped before a message arrives or the deadline
/// passes, you get [`RecvTimeoutError::Disconnected`].
///
/// The wake is classified from state alone — wakes are precise (the only
/// stamped wakers of this wait are a send, the last-sender drop, and the
/// timer; a stop wake unwinds out of `park_current` and never reaches
/// the classification), so: message queued → `Ok`; `senders == 0` →
/// `Disconnected`; neither → it was the timer → `Timeout`.
///
/// `Duration::ZERO` is a valid timeout: it parks until the immediately-
/// due timer is drained, then reports `Timeout` unless a message was
/// already queued.
/// `Duration::ZERO` is a valid timeout: it still gives any
/// already-queued message a chance to be returned, and only then
/// reports `Timeout`.
pub fn recv_timeout(&self, timeout: std::time::Duration) -> Result<T, RecvTimeoutError>
where
T: Send + 'static,
@@ -268,7 +340,7 @@ impl<T> Receiver<T> {
// Arm the timer after releasing the channel lock (insert takes the
// timers lock; never nest under a Channel lock). A send or even the
// timer itself may unpark us before we park — the RunningNotified
// timer itself may unpark us before we park; the runtime's wake
// protocol makes the park below return immediately in that case.
let deadline = crate::timer::deadline_from_now(timeout);
let target: std::sync::Arc<dyn crate::timer::TimerTarget> = self.inner.clone();
@@ -290,16 +362,23 @@ impl<T> Receiver<T> {
Err(RecvTimeoutError::Timeout)
}
/// Selective receive: remove and return the first queued message for which
/// `pred` holds, leaving the rest in arrival order. If no queued message
/// matches, parks and re-scans on every send (a selective receiver may park
/// on a *non-empty* queue). Returns `Err(RecvError)` only once the channel
/// is closed and no queued message matches.
/// Selective receive: find and return the first queued message for
/// which `pred` returns `true`, leaving every other message in the
/// queue untouched and in order. Useful when an actor's inbox mixes
/// message kinds and it wants to handle one kind out of turn, without
/// discarding the rest.
///
/// `pred` is run while the channel lock is held: keep it cheap and pure,
/// and do not call back into this channel from inside it. It is modelled as
/// `Fn` (not `FnMut`) deliberately — it is re-run from scratch on every
/// scan, so a stateful predicate would observe surprising re-counting.
/// If nothing queued matches, this blocks and re-checks every time a new
/// message arrives, the same way [`recv`](Self::recv) blocks on an empty
/// queue: a selective receiver can be waiting even while the queue holds
/// messages, just none that match yet. Returns [`RecvError`] only once
/// the channel is closed and still nothing matches.
///
/// `pred` runs while the channel is locked, so keep it cheap, side
/// effect free, and make sure it never calls back into this same
/// channel. It takes `&T` and is called fresh on every scan (not `FnMut`
/// with running state), so it should judge each message purely on its
/// own content.
pub fn recv_match<F>(&self, pred: F) -> Result<T, RecvError>
where
F: Fn(&T) -> bool,
@@ -331,7 +410,7 @@ impl<T> Receiver<T> {
g.parked_receiver = Some((me, crate::scheduler::begin_wait()));
crate::te!(crate::trace::Event::RecvPark(me));
}
// Release the lock before parking — the unparker will need it.
// Release the lock before parking: the unparker will need it.
crate::scheduler::park_current();
crate::te!(crate::trace::Event::RecvWake(match crate::actor::current_pid() {
Some(p) => p,
@@ -340,10 +419,12 @@ impl<T> Receiver<T> {
}
}
/// Non-blocking selective receive. `Ok(Some(v))` if a queued message
/// matched `pred` (removed, rest left in order), `Ok(None)` if the channel
/// is open but nothing matched, `Err(RecvError)` if closed and nothing
/// matched. Same predicate contract as [`recv_match`](Self::recv_match).
/// The non-blocking counterpart of [`recv_match`](Self::recv_match):
/// returns immediately either way. `Ok(Some(v))` if a queued message
/// matched `pred` (removed; the rest stay queued in order), `Ok(None)`
/// if the channel is open but nothing currently matches, `Err(RecvError)`
/// if the channel is closed and nothing matches. Same predicate contract
/// as `recv_match`.
pub fn try_recv_match<F>(&self, pred: F) -> Result<Option<T>, RecvError>
where
F: Fn(&T) -> bool,
@@ -363,8 +444,10 @@ impl<T> Receiver<T> {
Ok(None)
}
/// Non-blocking. `Ok(Some(v))` if a message was available, `Ok(None)` if
/// the channel is empty but open, `Err(RecvError)` if closed and drained.
/// The non-blocking counterpart of [`recv`](Self::recv): returns
/// immediately either way. `Ok(Some(v))` if a message was queued,
/// `Ok(None)` if the channel is open but currently empty, `Err(RecvError)`
/// if the channel is closed and the queue is drained.
pub fn try_recv(&self) -> Result<Option<T>, RecvError> {
let mut g = self.inner.lock();
if let Some(v) = g.queue.pop_front() {
@@ -379,18 +462,18 @@ impl<T> Receiver<T> {
}
// ---------------------------------------------------------------------------
// TimerTarget — the expiry half of recv_timeout
// TimerTarget: the expiry half of recv_timeout
// ---------------------------------------------------------------------------
impl<T: Send + 'static> crate::timer::TimerTarget for RawMutex<Inner<T>> {
fn on_timeout(&self, pid: Pid, epoch: u32) {
// Cancel the wait only if THIS wait (epoch match) is still
// registered. If a sender already took `parked_receiver`, the
// receiver is waking with a message — message wins, the timer
// receiver is waking with a message: message wins, the timer
// no-ops. If a later wait by the same receiver is registered, the
// epoch mismatches — stale entry, no-op. (The unpark_at would fail
// its word CAS in either case anyway; checking under the lock keeps
// the registration bookkeeping exact.)
// epoch mismatches: stale entry, no-op. (unpark_at would fail its
// internal check in either case anyway; checking under the lock
// keeps the registration bookkeeping exact.)
let unpark = {
let mut g = self.lock();
if g.parked_receiver == Some((pid, epoch)) {
@@ -400,7 +483,7 @@ impl<T: Send + 'static> crate::timer::TimerTarget for RawMutex<Inner<T>> {
false
}
};
// Unpark outside the channel lock — it may take the run-queue lock;
// Unpark outside the channel lock: it may take the run-queue lock;
// legal under a Channel lock, but pointless to nest.
if unpark {
crate::scheduler::unpark_at(pid, epoch);
@@ -409,7 +492,7 @@ impl<T: Send + 'static> crate::timer::TimerTarget for RawMutex<Inner<T>> {
}
// ---------------------------------------------------------------------------
// select — ready-index wait over multiple receivers
// select: ready-index wait over multiple receivers
// ---------------------------------------------------------------------------
pub(crate) mod sealed {
@@ -417,33 +500,34 @@ pub(crate) mod sealed {
}
impl<T> sealed::Sealed for Receiver<T> {}
/// An arm of a [`select`]. Implemented by [`Receiver`]; sealed, because the
/// registration contract below is part of the runtime's wake protocol.
/// An arm of a [`select`]: something you can wait on alongside other arms
/// and be told when it becomes ready. Implemented by [`Receiver`]; sealed
/// (cannot be implemented outside this crate), since the registration
/// contract below is part of the runtime's internal wake protocol.
///
/// Contract (all under the arm's own lock): `sel_register` checks-or-
/// registers atomically — if the arm is ready it does NOT register and
/// registers atomically. If the arm is ready it does not register and
/// returns `Ok(false)`; otherwise it publishes `(pid, epoch)` where its
/// wakers will find it and returns `Ok(true)`. "Ready" means a receive
/// would not park: a message is queued, or the arm is closed. `Err` means
/// would not block: a message is queued, or the arm is closed. `Err` means
/// the arm could not register at all (only fd arms can fail; channel
/// registration is infallible) — the wait must be retired and earlier
/// registration always succeeds), and the wait must be retired and earlier
/// eager-cleanup arms unregistered.
pub trait Selectable: sealed::Sealed {
#[doc(hidden)]
fn sel_register(&self, pid: Pid, epoch: u32) -> std::io::Result<bool>;
#[doc(hidden)]
fn sel_ready(&self) -> bool;
/// Remove this arm's `(pid, epoch)` registration if — and only if — it
/// is still in place. Default no-op: a losing channel arm's stale
/// registration is inert (its wakers die at the epoch CAS; the next
/// wait overwrites the slot). Fd arms override this: their staleness
/// poisons the fd (waiters entry + kernel-side ONESHOT registration)
/// and needs an eager cleanup pass.
/// Remove this arm's `(pid, epoch)` registration if, and only if, it is
/// still in place. Default no-op: a losing channel arm's stale
/// registration is harmless and self-cleans. Fd arms override this:
/// their staleness would otherwise leave the fd unusable for future
/// selects, so they need an eager cleanup pass.
#[doc(hidden)]
fn sel_unregister(&self, _pid: Pid, _epoch: u32) {}
/// Whether this arm requires the eager cleanup pass at all. Gates the
/// post-wake `sel_unregister` sweep so channel-only selects keep
/// today's zero-cancellation hot path.
/// post-wake `sel_unregister` sweep so channel-only selects keep their
/// cheap, cleanup-free path.
#[doc(hidden)]
fn sel_eager_cleanup(&self) -> bool {
false
@@ -470,38 +554,35 @@ impl<T> Selectable for Receiver<T> {
}
}
/// Park on every arm at once; return the index of the first ready one.
/// Wait on several channels at once and return the index of the first one
/// that is ready, instead of blocking on just one with [`Receiver::recv`].
///
/// "Ready" means a receive on that arm would not park: a message is queued,
/// or the arm is **closed** (so the caller's `try_recv` observes the
/// disconnect — a dead arm is an event, not a hang). The caller consumes the
/// arm itself, typically via [`Receiver::try_recv`]; single-receiver
/// channels guarantee nothing can steal the message in between.
/// "Ready" means a receive on that arm would not block: a message is
/// queued, or the arm is closed (so the caller's own `try_recv` observes
/// the disconnect: a dead arm is something to react to, not something to
/// hang on). `select` only tells you which arm is ready; read the actual
/// message yourself, typically with [`Receiver::try_recv`] on that arm.
///
/// A closed arm stays ready *forever*: once its disconnect has been
/// observed, drop it from the arm set — under priority order it would
/// otherwise win every subsequent call and starve every higher-indexed arm.
/// A closed arm stays ready forever. Once you have observed its disconnect,
/// drop it from the arm set you pass in next time: otherwise, under the
/// priority order below, it would win every subsequent call and starve
/// every arm listed after it.
///
/// Arms are scanned **in order**: index 0 is the highest priority, both on
/// the immediate-ready path and after a wake. This is a documented
/// guarantee (compose like BEAM receive clauses: put control channels
/// first), not an accident — and therefore there is NO fairness promise; a
/// saturated arm 0 starves arm 1 by design.
/// Arms are checked **in order**: index 0 is the highest priority, both
/// when checking immediately and after being woken. This is a deliberate,
/// documented guarantee, not an accident of implementation: put a control
/// or shutdown channel first so it is always noticed promptly. The
/// flip side is that there is **no fairness guarantee**: a busy arm 0 can
/// starve arm 1 indefinitely by design.
///
/// One actor may select on a channel and later `recv` on it (or select on
/// overlapping sets) freely. What stays illegal is what was always illegal:
/// two *different* actors receiving on one channel.
/// One actor can `select` on a channel and later plain `recv` on it (or
/// `select` again on an overlapping set of arms) with no restriction. What
/// stays illegal is what was always illegal for a channel: two *different*
/// actors receiving on the same one.
///
/// Built on the consuming-wake protocol (see slot_state.rs): all arms are
/// registered under one wait epoch; the winning wake consumes it, so losing
/// arms' registrations are inert and need no cancellation pass — they
/// self-clean at their wakers' failed CAS, or get overwritten by this
/// receiver's next wait on that channel.
///
/// Panics if `arms` is empty, when called outside an actor, or if an fd
/// arm fails to register (EBADF, EMFILE, a second waiter on one fd —
/// see [`try_select`] for the fallible form; channel-only selects cannot
/// fail).
/// Panics if `arms` is empty, if called outside an actor, or if an fd arm
/// fails to register (see [`try_select`] for the fallible form; a
/// channel-only `select` can never fail).
pub fn select(arms: &[&dyn Selectable]) -> usize {
match try_select(arms) {
Ok(i) => i,
@@ -509,9 +590,10 @@ pub fn select(arms: &[&dyn Selectable]) -> usize {
}
}
/// [`select`], fallible: `Err` when an arm fails to register (only fd
/// arms can — EBADF, EMFILE on the epoll set, or a second waiter on an
/// fd that already has one). On `Err` the wait is fully retired and no
/// The fallible form of [`select`]: `Err` when an arm fails to register.
/// Only fd arms can fail this way (for example, the file descriptor is
/// invalid, or something else is already waiting on it); a channel-only
/// select can never fail. On `Err` the wait is fully retired and no
/// registration is left behind: every arm registered before the failing
/// one has been unregistered.
pub fn try_select(arms: &[&dyn Selectable]) -> std::io::Result<usize> {
@@ -527,13 +609,12 @@ pub fn try_select(arms: &[&dyn Selectable]) -> std::io::Result<usize> {
}
// Stale fd registrations are not harmless (a losing fd arm's
// waiters entry poisons the fd with AlreadyExists and its
// kernel-side ONESHOT registration can fire arbitrarily late), so
// selects containing fd arms run an eager cleanup pass after the
// park — including when a terminal stop unwinds out of it, via
// the guard. Channel-only selects skip all of it: `eager` is
// false, the guard is disarmed, and the loser-arm self-cleaning
// story is unchanged.
// leftover registration can make the fd unusable for the next
// select until a kernel event happens to clear it), so selects
// containing fd arms run an eager cleanup pass after the park,
// including when a terminal stop unwinds out of it, via the guard.
// Channel-only selects skip all of it: `eager` is false, the guard
// is disarmed, and the loser-arm self-cleaning story is unchanged.
let eager = arms.iter().any(|a| a.sel_eager_cleanup());
let mut guard = UnregisterGuard { arms, me, epoch, armed: eager };
@@ -546,22 +627,22 @@ pub fn try_select(arms: &[&dyn Selectable]) -> std::io::Result<usize> {
drop(guard);
// Woken precisely: an arm's send (message) or last-sender drop
// (closure) consumed our epoch, and both leave their arm ready —
// return the first one, in priority order (which may be a
// (closure) is what woke us, and both leave their arm ready.
// Return the first ready one, in priority order (which may be a
// different, higher-priority arm than the one that woke us; its
// message stays queued and re-reports ready on the next call).
// Fd arms classify by a fresh zero-timeout poll, so they too are
// a pure function of state — independent of the registration the
// cleanup pass just removed.
// a pure function of current state, independent of the
// registration the cleanup pass just removed.
for (i, arm) in arms.iter().enumerate() {
if arm.sel_ready() {
return Ok(i);
}
}
// Unreachable by protocol (a stop wake unwinds out of
// park_current). Defensive: re-open the wait and re-register —
// stale own-registrations are overwritten (channels) or were
// removed by the cleanup pass above (fds).
// Unreachable in practice (a stop wake unwinds out of
// park_current before we get here). Defensive: re-open the wait
// and re-register; stale own-registrations are overwritten
// (channels) or were removed by the cleanup pass above (fds).
}
}
@@ -576,11 +657,11 @@ fn unregister_arms(arms: &[&dyn Selectable], me: Pid, epoch: u32) {
}
}
/// Stop-unwind twin of the explicit cleanup pass: a terminal stop unwinds
/// out of `park_current`, and a registered fd arm must not outlive its
/// actor (the generalization of `wait_fd`'s `Dereg`). Disarmed on the
/// normal path after the explicit pass runs; never armed when no fd arm
/// registered, keeping the channel-only path guard-free in effect.
// Stop-unwind twin of the explicit cleanup pass: a terminal stop unwinds
// out of `park_current`, and a registered fd arm must not outlive its
// actor. Disarmed on the normal path after the explicit pass runs; never
// armed when no fd arm is registered, keeping the channel-only path
// guard-free in effect.
struct UnregisterGuard<'a> {
arms: &'a [&'a dyn Selectable],
me: Pid,
@@ -596,20 +677,16 @@ impl Drop for UnregisterGuard<'_> {
}
}
/// The registration pass shared by [`select`] and [`select_timeout`]:
/// check-or-register each arm, in priority order, each atomically under its
/// own lock. Cross-arm atomicity is unnecessary: an arm becoming ready
/// right after its registration wakes the caller through the protocol (the
/// prep-to-park window is closed by RunningNotified).
///
/// `Ok(Some(i))` = arm `i` was ready, the pass stopped, and the wait has
/// been RETIRED (no park may follow): earlier arms hold live-epoch
/// registrations, so earlier *fd* arms are unregistered eagerly, then the
/// epoch is bumped, a landed notification eaten, and a pending stop
/// re-observed — without which a stale arm wake could fault a later
/// one-shot park. `Err` = an arm failed to register; identical unwind
/// (earlier fd arms unregistered, wait retired). `Ok(None)` = every arm
/// registered; the caller parks.
// The registration pass shared by `select` and `select_timeout`: check-or-
// register each arm, in priority order, each atomically under its own lock.
// Cross-arm atomicity is unnecessary: an arm becoming ready right after its
// registration still wakes the caller through the normal wake path.
//
// `Ok(Some(i))` = arm `i` was already ready, the pass stopped, and the wait
// has been fully retired (no park may follow): earlier fd arms are
// unregistered eagerly so none are left dangling. `Err` = an arm failed to
// register; same unwind (earlier fd arms unregistered, wait retired).
// `Ok(None)` = every arm registered successfully; the caller parks.
fn register_arms(
me: Pid,
epoch: u32,
@@ -633,10 +710,10 @@ fn register_arms(
Ok(None)
}
/// The [`select_timeout`] timer target: stateless, because precise wakes
/// make classification a pure function of channel state. The entry is
/// stamped with the select's epoch; if an arm already won, this unpark dies
/// at the word's epoch CAS (the no-cancellation convention in `timer.rs`).
// The `select_timeout` timer target: stateless, because a wake's cause can
// always be read back off plain channel state (an arm ready, or not). If
// an arm already won before the deadline, this timer's fire is simply
// ignored, the way any other stale wakeup is.
struct SelectTimeout;
impl crate::timer::TimerTarget for SelectTimeout {
fn on_timeout(&self, pid: Pid, epoch: u32) {
@@ -644,27 +721,21 @@ impl crate::timer::TimerTarget for SelectTimeout {
}
}
/// [`select`] with a deadline: returns `Some(index)` like `select`, or
/// `None` once `timeout` elapses with no arm ready.
/// Like [`select`], but gives up and returns `None` if no arm becomes
/// ready before `timeout` elapses.
///
/// All of `select`'s semantics carry over (priority order, closed arms
/// permanently ready, no fairness promise). The timeout is one more stamped
/// waker on the same wait epoch — nothing is registered in any arm for it,
/// so there is nothing to cancel or leak: an arm winning leaves the timer
/// entry to expire as a stale-epoch no-op; the timer winning leaves the
/// arms' registrations to self-clean exactly as a `select` loser's would.
/// All of `select`'s semantics carry over: arms are still checked in
/// priority order, a closed arm is still permanently ready, and there is
/// still no fairness guarantee across arms. A message that arrives at
/// essentially the same moment the deadline passes still wins, the same
/// way [`Receiver::recv_timeout`] resolves that race.
///
/// The wake is classified from state alone (wakes are precise): some arm
/// ready → `Some` of the first, in priority order; none ready → the timer
/// was the only remaining stamped waker → `None`. A message that races the
/// deadline resolves message-first, as `recv_timeout` does.
/// `Duration::ZERO` is a valid timeout: it still gives an already-ready arm
/// a chance to be reported before falling through to `None`.
///
/// `Duration::ZERO` is a valid timeout: it parks until the immediately-due
/// timer is drained, then reports `None` unless an arm was already ready.
///
/// Panics if `arms` is empty, when called outside an actor, or if an fd
/// arm fails to register (see [`try_select_timeout`] for the fallible
/// form; channel-only selects cannot fail).
/// Panics if `arms` is empty, if called outside an actor, or if an fd arm
/// fails to register (see [`try_select_timeout`] for the fallible form; a
/// channel-only select can never fail).
pub fn select_timeout(
arms: &[&dyn Selectable],
timeout: std::time::Duration,
@@ -677,9 +748,9 @@ pub fn select_timeout(
}
}
/// [`select_timeout`], fallible: `Err` when an arm fails to register
/// (only fd arms can). On `Err` the wait is fully retired and no
/// registration — arm-side or kernel-side — is left behind.
/// The fallible form of [`select_timeout`]: `Err` when an arm fails to
/// register (only fd arms can). On `Err` the wait is fully retired and no
/// registration is left behind on any arm.
pub fn try_select_timeout(
arms: &[&dyn Selectable],
timeout: std::time::Duration,
@@ -694,17 +765,16 @@ pub fn try_select_timeout(
return Ok(Some(i)); // ready now: the timer was never armed
}
// Arm the timer after the registration pass, outside every Channel
// lock (insert takes the timers lock).
// Arm the timer after the registration pass, outside every channel
// lock (inserting a timer takes the timers lock).
let deadline = crate::timer::deadline_from_now(timeout);
let target: std::sync::Arc<dyn crate::timer::TimerTarget> = std::sync::Arc::new(SelectTimeout);
crate::scheduler::insert_wait_timer(deadline, me, target, epoch);
// Same eager-cleanup story as `try_select`: the timer arm needs none
// (stateless, stale entries die at the epoch CAS), channel arms need
// none, fd arms do — and a timer win in particular leaves every fd
// arm's registration behind, which without this pass would poison
// those fds until a kernel event happened to fire.
// Same eager-cleanup story as `try_select`: a timer win in particular
// leaves every fd arm's registration behind, which without this pass
// would leave those fds unusable until a kernel event happened to
// clear them.
let eager = arms.iter().any(|a| a.sel_eager_cleanup());
let mut guard = UnregisterGuard { arms, me, epoch, armed: eager };
+31 -5
View File
@@ -182,7 +182,7 @@ use crate::channel::{channel, select, select_timeout, Receiver, RecvTimeoutError
use crate::monitor::{demonitor, monitor, Down, Monitor};
use crate::pid::Pid;
use crate::registry::{register_with, resolve_named_sender, RegisterError};
use crate::scheduler::{cancel_timer, request_stop, send_after_to, spawn, spawn_under};
use crate::scheduler::{cancel_timer, request_stop, send_after_to};
use crate::timer::TimerId;
use std::cell::Cell;
use std::collections::HashMap;
@@ -643,11 +643,17 @@ pub struct GenServerBuilder<G: GenServer> {
state: G,
infos: Vec<Receiver<G::Info>>,
supervisor: Option<Pid>,
stack_opts: crate::scheduler::SpawnOpts,
}
impl<G: GenServer> GenServerBuilder<G> {
pub fn new(state: G) -> Self {
GenServerBuilder { state, infos: Vec::new(), supervisor: None }
GenServerBuilder {
state,
infos: Vec::new(),
supervisor: None,
stack_opts: crate::scheduler::SpawnOpts::default(),
}
}
/// Add an out-of-band channel; messages arriving on it are dispatched to
@@ -665,6 +671,14 @@ impl<G: GenServer> GenServerBuilder<G> {
self
}
/// Stack shape for the server actor (RFC 019) — see
/// [`SpawnOpts`](crate::SpawnOpts). Useful for servers that recurse
/// deeply or call into FFI with large C frames.
pub fn stack_opts(mut self, opts: crate::scheduler::SpawnOpts) -> Self {
self.stack_opts = opts;
self
}
/// Spawn the server actor and hand back its [`GenServerRef`]. The server's
/// lifetime is governed by its refs, not by joining, so the backing join
/// handle is dropped.
@@ -686,10 +700,16 @@ impl<G: GenServer> GenServerBuilder<G> {
/// under the name before returning.
fn spawn_server(self) -> GenServerRef<G> {
let (tx, rx) = channel::<Envelope<G>>();
let GenServerBuilder { state, infos, supervisor } = self;
let GenServerBuilder { state, infos, supervisor, stack_opts } = self;
let handle = match supervisor {
Some(sup) => spawn_under(sup, move || server_loop::<G>(rx, state, infos)),
None => spawn(move || server_loop::<G>(rx, state, infos)),
Some(sup) => {
crate::scheduler::spawn_under_with(sup, stack_opts, move || {
server_loop::<G>(rx, state, infos)
})
}
None => crate::scheduler::spawn_with(stack_opts, move || {
server_loop::<G>(rx, state, infos)
}),
};
GenServerRef { tx, pid: handle.pid() }
}
@@ -758,6 +778,12 @@ impl<G: GenServer> NamedGenServerBuilder<G> {
self
}
/// Stack shape for the server actor (see [`GenServerBuilder::stack_opts`]).
pub fn stack_opts(mut self, opts: crate::scheduler::SpawnOpts) -> Self {
self.builder = self.builder.stack_opts(opts);
self
}
/// Spawn the server and bind its name in one step. Fallible: returns
/// [`RegisterError::NameTaken`] if the name is already held by a different
/// live server.
+15 -2
View File
@@ -71,7 +71,7 @@
use crate::channel::{channel, select, Receiver, Sender};
use crate::pid::Pid;
use crate::scheduler::{cancel_timer, send_after_to, spawn as spawn_actor};
use crate::scheduler::{cancel_timer, send_after_to};
use crate::timer::TimerId;
use std::collections::{HashMap, VecDeque};
use std::marker::PhantomData;
@@ -434,8 +434,21 @@ impl<M: Machine> GenStatemRef<M> {
///
/// Panics if called outside `Runtime::run()`.
pub fn spawn<M: Machine>(machine: M) -> GenStatemRef<M> {
spawn_with(crate::scheduler::SpawnOpts::default(), machine)
}
/// [`spawn`] with per-actor stack shape overrides (RFC 019) for the machine's
/// actor — see [`SpawnOpts`](crate::SpawnOpts). gen_statem has no builder
/// (its one-shot `spawn(machine)` shape predates RFC 019), so the opts ride
/// a `_with` variant like the scheduler's own spawns.
///
/// Panics if called outside `Runtime::run()`.
pub fn spawn_with<M: Machine>(
opts: crate::scheduler::SpawnOpts,
machine: M,
) -> GenStatemRef<M> {
let (tx, rx) = channel::<M::Ev>();
let handle = spawn_actor(move || statem_loop(rx, machine));
let handle = crate::scheduler::spawn_with(opts, move || statem_loop(rx, machine));
GenStatemRef { tx, pid: handle.pid() }
}
+234 -83
View File
@@ -1,26 +1,85 @@
//! RFC 016 — runtime introspection (Chunk 1: the read primitive).
//! Inspect what is running right now: which actors exist, what state each one
//! is in, and how they are related.
//!
//! A synchronous, internal read of the slab that returns *owned* data. This is
//! the mechanism the whole RFC hangs off: tests, the future observer
//! gen_server (Chunk 4), and a later control plane (RFC 003) are all consumers
//! of [`snapshot`] / [`actor_info`], never of the runtime internals directly.
//! This is the tool for questions like "is my server still alive", "how many
//! actors are currently parked waiting on something", or "what does the spawn
//! tree look like". It is meant for debugging, test assertions, a health check
//! endpoint, or a monitoring dashboard: anywhere you want to look at the
//! runtime from the outside without stopping it or coupling your code to its
//! internals.
//!
//! ## Consistency (DECISION D2 — per-slot tearing, `ps` semantics)
//! Three entry points, in order of scope:
//!
//! [`snapshot`] is point-in-time and mildly racy *across* actors: each slot's
//! scheduling state is a lock-free word load, so an actor reported `Running`
//! may already be `Parked`, and an actor can die mid-scan. This is the cheap,
//! useful model (a coherent stop-the-world cut is expensive and rarely wanted).
//! [`actor_info`] is coherent for the single actor it names.
//! - [`snapshot`] returns every actor that currently exists, as a plain
//! owned `Vec`, so you can filter, count, or search it however you like.
//! - [`actor_info`] returns a coherent view of exactly one actor, by pid.
//! Cheaper than filtering a whole snapshot down to one entry, and more
//! precise (see "Consistency" below).
//! - [`tree`] returns the same actors as [`snapshot`], folded into a
//! parent/child forest that mirrors who spawned whom.
//!
//! ## Locking
//! ```
//! use smarm::{actor_info, channel, run, snapshot, spawn, ActorState};
//!
//! The lock order is **Leaf → Channel, at most one of each** (`raw_mutex.rs`);
//! cold locks, the registry, and the free list are all Leaves, so we may never
//! hold two at once. The read is therefore phased: first a single registry-leaf
//! pass for names and mailbox depth (the per-channel length read is a Channel
//! lock taken under that Leaf — legal), released before the slab scan takes any
//! per-slot cold Leaf.
//! run(|| {
//! let (ready_tx, ready_rx) = channel::<()>();
//! let (gate_tx, gate_rx) = channel::<()>();
//!
//! let worker = spawn(move || {
//! ready_tx.send(()).unwrap();
//! gate_rx.recv().unwrap(); // blocks here until released
//! });
//! ready_rx.recv().unwrap();
//!
//! // `snapshot` sees every actor, including this one and the worker.
//! let snap = snapshot();
//! assert!(snap.actors.len() >= 2);
//!
//! // `actor_info` gives a coherent view of just the worker. It is
//! // blocked on the gate channel, so it must be Parked.
//! let pid = worker.pid();
//! let info = actor_info(pid).expect("worker is still alive");
//! assert_eq!(info.state, ActorState::Parked);
//!
//! gate_tx.send(()).unwrap();
//! worker.join().unwrap();
//!
//! // Once joined, the pid no longer names a live actor.
//! assert!(actor_info(pid).is_none());
//! });
//! ```
//!
//! ## Consistency
//!
//! [`snapshot`] is not a single atomic pause-the-world freeze: it walks every
//! actor's state one after another, so it is a series of independent,
//! cheap, lock-free reads rather than one coherent moment in time. Between
//! reading actor A and actor B, either one can change state, and an actor can
//! even finish and disappear mid-scan. In practice this is exactly what you
//! want: a coherent stop-the-world snapshot would mean pausing every actor in
//! the runtime just to look at it, which is expensive and rarely necessary
//! for a dashboard, a test assertion, or a debugging session.
//!
//! [`actor_info`], in contrast, is coherent for the one actor it names: all of
//! its fields describe the same instant for that actor, because a single
//! actor's data cannot tear the way a scan across many actors can.
//!
//! ## Implementation notes
//!
//! These details matter if you are working on smarm itself; they are not part
//! of the public contract.
//!
//! The read never stops the scheduler and never holds a lock across the whole
//! scan. Each actor's scheduling state is a single lock-free word load
//! (hence the possible tearing described above). Reading the rest of an
//! actor's cold data (its supervisor, monitors, links, and so on) takes a
//! brief per-actor lock, just long enough to copy those fields out; nothing
//! is held across actors. Locking follows the crate-wide rule that at most
//! one "leaf" lock (a per-actor lock, the registry lock, or the free list
//! lock) is held at a time, with no leaf lock held while acquiring another.
//! The read is phased accordingly: first one pass over the registry to
//! collect every actor's registered names and mailbox depth, released before
//! the per-actor scan begins.
use crate::pid::Pid;
use crate::registry::MailboxInfo;
@@ -31,15 +90,28 @@ use crate::slot_state::{
};
use std::collections::HashMap;
/// Snapshot wire-format version (DECISION D1). [`RuntimeSnapshot`] is treated as
/// a stable type from day one: it becomes the observer protocol (Chunk 4) and
/// crosses a version boundary the moment a remote observer attaches (RFC 011),
/// so the version travels with the data from the start.
/// The format version carried by every [`RuntimeSnapshot`] and
/// [`RuntimeTree`], as [`RuntimeSnapshot::format_version`] /
/// [`RuntimeTree::format_version`]. If you serialize a snapshot (for example
/// to send it somewhere else, or to compare snapshots taken with different
/// versions of smarm) check this field: a change in its value means the shape
/// of [`ActorInfo`] or its neighbors has changed and old and new snapshots
/// should not be assumed compatible. If you only ever read a snapshot
/// in-process in the same version of smarm that produced it, you can ignore
/// this field.
pub const SNAPSHOT_FORMAT_VERSION: u16 = 1;
/// Fine-grained scheduling state, mapped from the packed slot word with no new
/// storage. `RunningNotified` collapses into `Notified` — a wake landed while
/// the actor was on-CPU and it will re-queue when it yields.
/// What an actor is doing right now, from the scheduler's point of view.
///
/// - `Queued`: runnable, waiting for a scheduler thread to pick it up.
/// - `Running`: currently executing on a scheduler thread.
/// - `Notified`: was running and got woken up (for example, a message
/// arrived) before it had a chance to yield or park; it will be re-queued
/// as soon as it does.
/// - `Parked`: blocked, waiting on something such as a channel receive, a
/// mutex, a timer, or an IO event.
/// - `Done`: has finished (returned or panicked) but its slot has not been
/// reclaimed for reuse yet, so it is still visible to introspection.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ActorState {
Queued,
@@ -49,8 +121,8 @@ pub enum ActorState {
Done,
}
/// Classify a packed state word. `None` for a Vacant slot (skipped by the scan)
/// — the only state that is not an actor.
/// Classify a packed state word. `None` for a Vacant slot (skipped by the
/// scan): a vacant slot holds no actor at all, live or done.
fn classify(w: u64) -> Option<ActorState> {
Some(match word_state(w) {
ST_QUEUED => ActorState::Queued,
@@ -62,61 +134,104 @@ fn classify(w: u64) -> Option<ActorState> {
})
}
/// Owned, point-in-time view of one actor — no borrows of runtime internals, so
/// it is safe to hand to any consumer.
/// An owned, self-contained view of one actor at (approximately) one moment.
/// It borrows nothing from the runtime, so you can keep it, send it
/// elsewhere, or print it long after the actor it describes has changed
/// state or even exited.
#[derive(Debug, Clone)]
pub struct ActorInfo {
pub pid: Pid,
/// Registered names, inverted from the registry (usually 0 or 1).
/// Names this actor is currently registered under (see the
/// [`registry`](crate::registry) module). Usually empty or one name;
/// an actor can have more if it registered several.
pub names: Vec<&'static str>,
pub state: ActorState,
/// Spawn-time parent edge (DECISION D9): `spawn_under` sets it to the
/// supervisor, plain `spawn` to the spawning actor — so it is parentage,
/// not necessarily a supervision relationship. `ROOT_PID` for the run's
/// root actor and for `Done` tombstones (whose `Actor` is already gone).
/// The actor that spawned this one: whoever called `spawn` or
/// `spawn_under` to create it. This is a parentage record, not
/// necessarily a supervision relationship: `spawn_under` records the
/// supervisor you asked for, while plain `spawn` records the spawning
/// actor itself, whether or not it supervises anything. It is the
/// runtime's root pid for the run's own root actor, and for a `Done`
/// actor whose bookkeeping has already been cleared.
pub supervisor: Pid,
pub trap_exit: bool,
pub monitors: u32,
pub links: u32,
pub joiners: u32,
/// Queued messages summed over the actor's *published* channels (register /
/// install / spawn_addr / gen_server). 0 for an actor that holds only a
/// private `channel()` receiver — those are invisible to the registry.
/// Messages currently queued and not yet delivered, summed across every
/// channel this actor has published (via `register`, `install`,
/// `spawn_addr`, or starting a gen_server). This is 0 for an actor that
/// only holds a private, unpublished `channel()` receiver, since nothing
/// outside the actor can see that channel exists.
pub mailbox_depth: u32,
/// Timeslice overruns tallied for this incarnation (RFC 016 Chunk 2): how
/// many times the actor was preempted for exceeding its slice. Resets on
/// restart (per-incarnation, D7).
/// How many times this actor has been preempted for running past its
/// scheduling timeslice. Counts only since the actor's current start (a
/// supervisor restart begins a fresh count).
pub overruns: u64,
/// Messages this actor has received (dequeued) this incarnation (RFC 016
/// Chunk 2) — answers "is this actor a hotspot / draining slower than its
/// mailbox fills." Counts received, not sent (D4). Per-incarnation (D7).
/// How many messages this actor has received (taken off its inbox), since
/// its current start. Useful for spotting an actor whose mailbox is
/// filling up faster than it can drain it: compare this against
/// `mailbox_depth` over time.
pub messages_received: u64,
/// Approximate on-CPU cycles this incarnation has consumed (RFC 016 Chunk 2)
/// — a reductions-like work metric for relative comparison. Always 0 unless
/// the `budget-accounting` feature is enabled (it costs an RDTSC per resume,
/// D6). Per-incarnation (D7).
/// Approximate CPU cycles this actor has spent running, since its current
/// start. A relative measure for comparing actors against each other, not
/// an absolute or wall-clock figure. Always 0 unless the crate's
/// `budget-accounting` feature is enabled, since measuring it costs a
/// timestamp read on every resume.
pub budget_cycles: u64,
/// RFC 019 §8 — this actor's stack, as the runtime sees it. All fields
/// are lock-free atomic reads, coherent for this incarnation via the
/// same generation check as the counters above. Exact RSS is
/// deliberately absent: `mincore` is debug tooling, never a runtime
/// path.
pub stack: StackInfo,
}
/// A whole-runtime snapshot. See the module docs for the D2 tearing model.
/// RFC 019 §8 — per-actor stack introspection. Sizes are page-rounded, as
/// [`Stack::new`](crate::stack::Stack::new) rounds them.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct StackInfo {
/// Usable stack size ([`SpawnOpts::stack_reserve`]
/// (crate::SpawnOpts::stack_reserve) or the Config/default).
pub reserve: usize,
/// PROT_NONE guard below the usable region.
pub guard: usize,
/// Sampled high-water depth in bytes: `top − lowest saved sp`. Sampled,
/// not exact — the context save at yields/parks/preemptions is the
/// sampler (RFC 019 §2), so a spike the actor never yielded inside is
/// invisible. 0 depth means "never descheduled at any depth", not
/// "never ran".
pub depth_high_water: usize,
/// Parks on this incarnation since its last shrink (or since install if
/// it has never shrunk) — the §3 cooldown counter, live.
pub parks_since_shrink: u32,
/// §3 shrinks performed on this incarnation.
pub shrinks: u32,
}
/// A snapshot of every actor in the runtime at (approximately) one moment.
/// See the module docs' "Consistency" section for what "approximately" means
/// here.
#[derive(Debug, Clone)]
pub struct RuntimeSnapshot {
pub format_version: u16,
pub actors: Vec<ActorInfo>,
}
/// Snapshot every live (and `Done`-but-not-yet-reclaimed) actor on the slab.
/// O(n) over the slot table, running with preemption disabled (like every
/// runtime primitive) but holding no lock across the scan. Panics outside
/// `Runtime::run()`; callable from actor code and the run thread.
/// Every actor that currently exists: running, queued, parked, or finished
/// but not yet cleaned up. Cheap and lock-free per actor; see the module
/// docs for what "approximately one moment" means for the result as a whole.
/// Panics if called outside [`run`](crate::run).
pub fn snapshot() -> RuntimeSnapshot {
with_runtime(|inner| {
// Phase A: one registry-leaf pass for names + mailbox depth, released
// before any cold leaf (no two Leaves at once).
// First pass: one registry lock to collect names + mailbox depth for
// every actor, released before touching any per-actor lock below.
let mail = inner.registry.lock().introspect_map();
// Phase B: lock-free slab scan; per-slot cold leaf only to copy cold
// fields. Tearing across slots is intentional (D2).
// Second pass: walk the actor table. Each actor's scheduling state is
// a lock-free word load; only copying its other fields takes a brief
// per-actor lock. Tearing across actors is expected here (see the
// module docs' "Consistency" section).
let mut actors = Vec::new();
for (idx, slot) in inner.slots.iter().enumerate() {
let idx = idx as u32;
@@ -128,8 +243,27 @@ pub fn snapshot() -> RuntimeSnapshot {
})
}
/// Coherent view of a single actor, or `None` if the pid is stale, out of
/// range, or names a Vacant slot.
/// A coherent view of exactly one actor, or `None` if `pid` does not name a
/// currently-live entry: it is stale (that actor has already exited and its
/// slot was reused by another), out of range, or was never a real pid at
/// all. Unlike [`snapshot`], every field of the result describes the same
/// instant, since there is only one actor to read.
/// The stack shape `(reserve, guard)` of a live actor, page-rounded — the
/// RFC 019 introspection surface's first field (depth sampling and shrink
/// counters land with the shrink machinery). `None` if `pid` no longer names
/// a live actor. Takes the actor's cold lock briefly; debugging/assertion
/// use, not a hot-path call.
pub fn stack_shape(pid: Pid) -> Option<(usize, usize)> {
with_runtime(|inner| {
let slot = inner.slot_at(pid)?;
let cold = slot.cold.lock();
if slot.generation() != pid.generation() {
return None;
}
cold.actor.as_ref().map(|a| a.stack.shape())
})
}
pub fn actor_info(pid: Pid) -> Option<ActorInfo> {
with_runtime(|inner| {
let slot = inner.slot_at(pid)?;
@@ -141,11 +275,12 @@ pub fn actor_info(pid: Pid) -> Option<ActorInfo> {
})
}
/// Build one `ActorInfo` for slot `idx`, or `None` if Vacant or
/// racing-reclaimed. State is classified from a lock-free word load (the torn
/// read); the cold lock then pins the generation (reclaim bumps it under that
/// same lock) so the cold fields are coherent for this incarnation. `mail` is
/// this slot's registry entry, if any.
/// Build one `ActorInfo` for slot `idx`, or `None` if the slot is empty or
/// was reclaimed while this read was in progress. The scheduling state comes
/// from a lock-free word load (the source of the tearing described in the
/// module docs); the per-actor lock then confirms the actor has not since
/// exited and been replaced, so the rest of the fields are coherent for this
/// exact actor. `mail` is this slot's registry entry, if any.
fn read_slot(slot: &Slot, idx: u32, mail: Option<&MailboxInfo>) -> Option<ActorInfo> {
let w = slot.state_word();
let state = classify(w)?;
@@ -153,10 +288,11 @@ fn read_slot(slot: &Slot, idx: u32, mail: Option<&MailboxInfo>) -> Option<ActorI
let pid = Pid::new(idx, gen);
let cold = slot.cold.lock();
// If the generation moved between the lock-free load and acquiring the cold
// lock, the slot was reclaimed (and maybe reused) — drop it rather than mix
// one incarnation's state with another's cold data. (ps semantics: a racing
// actor may simply be missed mid-scan.)
// If the generation moved between the lock-free load and acquiring the
// per-actor lock, this actor exited (and the slot may already hold a new
// one). Drop it rather than mix one actor's state with another's data; a
// racing actor may simply be missed by this scan, which is expected (see
// the module docs' "Consistency" section).
if word_gen(slot.state_word()) != gen {
return None;
}
@@ -172,7 +308,15 @@ fn read_slot(slot: &Slot, idx: u32, mail: Option<&MailboxInfo>) -> Option<ActorI
let joiners = cold.waiters.len() as u32;
drop(cold);
// Counters are hot-region atomics, read lock-free (RFC 016 Chunk 2).
// Counters are plain atomics, read lock-free.
let (reserve, guard, top, hwm, parks_since_shrink, shrinks) = slot.stack_introspect();
let stack = StackInfo {
reserve,
guard,
depth_high_water: top.saturating_sub(hwm),
parks_since_shrink,
shrinks,
};
let overruns = slot.overruns();
let messages_received = slot.messages_received();
let budget_cycles = slot.budget_cycles();
@@ -198,43 +342,50 @@ fn read_slot(slot: &Slot, idx: u32, mail: Option<&MailboxInfo>) -> Option<ActorI
overruns,
messages_received,
budget_cycles,
stack,
})
}
// ---------------------------------------------------------------------------
// Chunk 3 — tree view (pure derivation over a Chunk-1 snapshot)
// Tree view: a pure derivation over a snapshot
// ---------------------------------------------------------------------------
/// One node in the parentage forest. `children` are the actors whose recorded
/// parent edge points at this node's pid.
/// One node in the parentage forest returned by [`tree`]. `children` are the
/// actors whose recorded parent (see [`ActorInfo::supervisor`]) points at
/// this node's actor.
#[derive(Debug, Clone)]
pub struct TreeNode {
pub info: ActorInfo,
/// The actor's recorded parent was absent from the snapshot (already
/// Done/Vacant, or itself a tombstone), so it was re-rooted under the forest
/// sentinel rather than dropped — the tree stays total (DECISION D8).
/// True if this actor's recorded parent was not found in the snapshot
/// (it had already exited, or was itself missing), so this node was
/// placed at the top of the forest instead of being dropped. This keeps
/// every actor in the snapshot visible somewhere in the tree, even one
/// whose parent is gone.
pub orphaned: bool,
pub children: Vec<TreeNode>,
}
/// The parentage forest. Roots are actors parented at `ROOT_PID` (genuine
/// roots) plus re-rooted orphans. The edge is *spawned-by / parent*, not
/// necessarily supervision (DECISION D9) — see [`ActorInfo::supervisor`].
/// The parentage forest: every actor from a snapshot, arranged by who spawned
/// whom. Roots are actors with no parent in the snapshot (including the
/// run's own root actor) plus any orphaned actors (see [`TreeNode::orphaned`]).
/// This mirrors spawn parentage, not necessarily a supervision tree; see
/// [`ActorInfo::supervisor`].
#[derive(Debug, Clone)]
pub struct RuntimeTree {
pub format_version: u16,
pub roots: Vec<TreeNode>,
}
/// Take a live [`snapshot`] and fold it into the parentage forest.
/// Take a fresh [`snapshot`] and fold it into the parentage forest.
pub fn tree() -> RuntimeTree {
tree_from(snapshot())
}
/// Fold an existing snapshot into a forest by grouping each actor under its
/// parent pid — a single O(n) pass, no new reads. Exposed separately so a
/// consumer that already holds a snapshot (or a synthetic one, in tests) can
/// derive the tree without a second scan.
/// Fold an existing snapshot into a parentage forest by grouping each actor
/// under its parent, without taking a new snapshot. Useful if you already
/// have one (for example, one built in a test, or one you took earlier and
/// want to inspect again) and want the tree view of it without re-reading
/// the runtime.
pub fn tree_from(snap: RuntimeSnapshot) -> RuntimeTree {
let RuntimeSnapshot { format_version, actors } = snap;
@@ -254,7 +405,7 @@ pub fn tree_from(snap: RuntimeSnapshot) -> RuntimeTree {
children_of.entry(parent).or_default().push(i);
} else {
// Parent is the forest sentinel (genuine root) or absent from the
// snapshot (orphan, D8) — either way a root of the forest.
// snapshot (orphan): either way, a root of the forest.
orphaned[i] = parent != ROOT_PID;
roots.push(i);
}
+166 -238
View File
@@ -13,44 +13,68 @@
//! leaves the actor, no copying through an intermediary thread. Built on
//! these are the conveniences `read(fd, &mut buf)` and `write(fd, &buf)`.
//!
//! Architecture
//! ============
//! Per `run()`, two OS threads:
//! - **epoll thread**: owns the epollfd. Loops in `epoll_wait`. On a
//! ready fd, pushes `Completion::FdReady { pid, fd, events }` to the
//! shared completion queue and writes the scheduler-wake pipe. On the
//! shutdown pipe (also registered in epollfd), exits.
//! - **pool thread**: blocks on the request mpsc. Runs the closure
//! inside `catch_unwind`, pushes `Completion::Blocking { pid, result }`,
//! writes the scheduler-wake pipe.
//! Architecture (RFC 018: driver-enqueues)
//! =======================================
//! Per `run()`, two OS threads, each a *producer* behind the runtime's
//! two-call contract — make the actor runnable (`unpark_at`, whose enqueue
//! tail wakes a parked scheduler), nothing else:
//!
//! Both threads share a single `completions: Arc<Mutex<VecDeque<Completion>>>`
//! and the same scheduler-wake pipe.
//! - **epoll thread**: owns `epoll_wait` on the epollfd. On a ready fd it
//! removes the parked waiter from the shared `waiters` map and DELs the
//! fd (both under the waiters lock — see below), then unparks the
//! actor directly. On the shutdown pipe (also registered in the
//! epollfd), exits.
//! - **pool thread**: blocks on the request mpsc. Runs the closure inside
//! `catch_unwind`, stashes the result in the actor's slot
//! (`pending_io_result`, under the cold lock, generation-checked),
//! decrements the runtime's `io_outstanding`, and unparks the actor.
//!
//! `epoll_ctl` (register/unregister fd interest) is called by the
//! scheduler thread *directly* on the epollfd. That's well-defined per
//! `epoll_ctl(2)`: a thread may be calling `epoll_wait` on the epollfd
//! while another thread calls `epoll_ctl`. Avoids needing a second mpsc
//! and a second wake mechanism.
//! There is no shared completion queue and no wake pipe: each producer
//! routes its own completion, so the whole byte-vs-completion visibility
//! discipline of the drain era — and the stranded-completion hazards it
//! defended against — is unrepresentable. Producers reach the runtime
//! through a `Weak<RuntimeInner>`: upgraded per completion (the path is
//! syscall-bound; the refcount op is noise) and avoiding an Arc cycle
//! through `RuntimeInner::io`.
//!
//! `epoll_ctl` (register fd interest) is called by the scheduler thread
//! directly on the epollfd. That's well-defined per `epoll_ctl(2)`: a
//! thread may be calling `epoll_wait` on the epollfd while another thread
//! calls `epoll_ctl`.
//!
//! Epoll mode
//! ==========
//! Level-triggered with EPOLLONESHOT. After a wakeup the kernel
//! auto-disarms the fd, so we never get two wakeups for one
//! `wait_readable` call. The scheduler explicitly `EPOLL_CTL_DEL`s the fd
//! on completion to free the slot for re-registration. Net effect: each
//! `wait_readable` call. The epoll thread explicitly `EPOLL_CTL_DEL`s the
//! fd on readiness to free the slot for re-registration. Net effect: each
//! `wait_readable(fd)` is one ADD, one wakeup, one DEL — symmetric and
//! stateless between calls.
//!
//! ## The waiters lock is the ADD/DEL serialization
//!
//! Registration (scheduler thread: check-vacant, defensive DEL, ADD,
//! insert) and readiness consumption (epoll thread: remove, DEL) each run
//! entirely under the `waiters` mutex. This is what makes the
//! oneshot-rearm race unrepresentable: a woken actor re-registering the
//! same fd cannot interleave with the epoll thread's DEL for the *previous*
//! registration — whichever takes the lock second sees a consistent
//! kernel-side state. Lock order: `io` (the runtime's outer mutex, held by
//! scheduler-side callers) → `waiters` → slot/queue leaves via `unpark_at`.
//! The epoll thread takes `waiters` without `io` — it must never take
//! `io`, both for lock-order hygiene and because teardown holds `io` while
//! joining it.
//!
//! Fd hygiene
//! ==========
//! An actor stopped while waiting on an fd unwinds out of `wait_fd`'s park;
//! a drop guard there (armed after a successful register, forgotten on a
//! normal wake) removes the `waiters` entry iff it is still that wait's
//! `(pid, epoch)` and only then `EPOLL_CTL_DEL`s the fd — an entry already
//! consumed by a racing `FdReady` means the fd may carry someone else's
//! fresh registration, which must be left alone. `epoll_register` keeps a
//! defensive bare DEL before ADD as belt-and-braces.
//! normal wake) calls [`IoThread::cancel_waiter`], which removes the
//! `waiters` entry iff it is still that wait's `(pid, epoch)` and only then
//! `EPOLL_CTL_DEL`s the fd — an entry already consumed by the epoll thread
//! means the fd may carry someone else's fresh registration, which must be
//! left alone. `epoll_register` keeps a defensive bare DEL before ADD as
//! belt-and-braces.
//!
//! Buffers used with `read`/`write` should be on fds opened with
//! `O_NONBLOCK`. If they aren't, the syscall may block the scheduler
@@ -68,13 +92,14 @@
//! they have no equivalent panic-propagation path.
use crate::pid::Pid;
use crate::runtime::RuntimeInner;
use std::any::Any;
use std::collections::{HashMap, VecDeque};
use std::collections::HashMap;
use std::io;
use std::os::fd::RawFd;
use std::panic;
use std::sync::mpsc;
use std::sync::{Arc, Mutex};
use std::sync::atomic::Ordering;
use std::sync::{mpsc, Arc, Mutex, Weak};
use std::thread::JoinHandle as OsJoinHandle;
// ---------------------------------------------------------------------------
@@ -86,42 +111,29 @@ use std::thread::JoinHandle as OsJoinHandle;
pub type IoResult = Result<Box<dyn Any + Send>, Box<dyn Any + Send>>;
struct Request {
/// The submitter's park-epoch — carried through to the `Blocking`
/// completion so the wake is epoch-matched.
/// The submitter's park-epoch — the eventual wake is epoch-matched.
epoch: u32,
pid: Pid,
/// The work to perform. Returns the wire-form result directly.
work: Box<dyn FnOnce() -> IoResult + Send>,
}
/// Completion message from either IO thread back to the scheduler.
pub enum Completion {
/// A `block_on_io` closure has finished (Ok = return value, Err = panic
/// payload).
Blocking { pid: Pid, epoch: u32, result: IoResult },
/// An fd registered via `wait_readable`/`wait_writable` is ready. The
/// scheduler looks up the parked pid in `waiters`, unparks it, and
/// removes the entry. `pid` isn't in this variant because the epoll
/// thread doesn't have access to the `waiters` map; the scheduler
/// thread owns that.
FdReady { fd: RawFd, events: u32 },
}
/// The parked-waiter map, shared between scheduler-side registration and
/// the epoll thread's readiness consumption. See the module docs on why
/// this single lock is the ADD/DEL serialization.
type Waiters = Arc<Mutex<HashMap<RawFd, (Pid, u32)>>>;
// ---------------------------------------------------------------------------
// IoThread — created per `run()`, owned by `SchedulerState`.
// IoThread — created per `run()`, owned by `RuntimeInner::io`.
// ---------------------------------------------------------------------------
pub struct IoThread {
// ----- Channels & queues -----
/// Submission queue into the blocking-work pool.
tx: mpsc::Sender<Request>,
/// Shared completion queue, fed by both the pool and the epoll thread.
completions: Arc<Mutex<VecDeque<Completion>>>,
/// Pipe the scheduler polls in its idle path. Both IO threads write to
/// `wake_write` after pushing a completion.
wake_read: RawFd,
wake_write: RawFd,
/// One parked actor per registered fd. Populated by `epoll_register`,
/// consumed by the epoll thread on readiness or `cancel_waiter` on an
/// unwound wait.
waiters: Waiters,
// ----- Epoll machinery -----
@@ -133,39 +145,25 @@ pub struct IoThread {
/// shutdown.
shutdown_read: RawFd,
shutdown_write: RawFd,
/// One parked actor per registered fd. Populated by `wait_readable` /
/// `wait_writable` and drained by the scheduler when a `FdReady`
/// completion is processed.
pub waiters: HashMap<RawFd, (Pid, u32)>,
// ----- Threads -----
pool_thread: Option<OsJoinHandle<()>>,
epoll_thread: Option<OsJoinHandle<()>>,
/// Number of `block_on_io` requests in-flight. Used by the scheduler's
/// idle path to decide whether to wait on the pipe or exit. Fd waits
/// are not counted here; they're counted by `waiters.len()`.
pub outstanding: u32,
}
impl IoThread {
pub fn start() -> io::Result<Self> {
// Scheduler-facing wake pipe.
let (wake_read, wake_write) = make_pipe()?;
// Pool submission channel + shared completion queue.
/// Start the pool and epoll threads. `rt` is the producers' route back
/// into the runtime (slot table + unpark protocol); a `Weak` so the
/// `RuntimeInner → IoThread → RuntimeInner` cycle never forms.
pub(crate) fn start(rt: Weak<RuntimeInner>) -> io::Result<Self> {
// Pool submission channel.
let (tx, rx) = mpsc::channel::<Request>();
let completions: Arc<Mutex<VecDeque<Completion>>> =
Arc::new(Mutex::new(VecDeque::new()));
let waiters: Waiters = Arc::new(Mutex::new(HashMap::new()));
// Epoll machinery.
let epollfd = unsafe { libc::epoll_create1(libc::EPOLL_CLOEXEC) };
if epollfd < 0 {
// Best-effort fd cleanup before bailing.
unsafe {
libc::close(wake_read);
libc::close(wake_write);
}
return Err(io::Error::last_os_error());
}
@@ -174,8 +172,6 @@ impl IoThread {
Err(e) => {
unsafe {
libc::close(epollfd);
libc::close(wake_read);
libc::close(wake_write);
}
return Err(e);
}
@@ -202,42 +198,37 @@ impl IoThread {
libc::close(epollfd);
libc::close(shutdown_read);
libc::close(shutdown_write);
libc::close(wake_read);
libc::close(wake_write);
}
return Err(e);
}
// Spawn pool thread.
let pool_comps = completions.clone();
let pool_rt = rt.clone();
let pool_thread = std::thread::Builder::new()
.name("smarm-io-pool".into())
.spawn(move || pool_loop(rx, pool_comps, wake_write))?;
.spawn(move || pool_loop(rx, pool_rt))?;
// Spawn epoll thread.
let epoll_comps = completions.clone();
let epoll_waiters = waiters.clone();
let epoll_thread = std::thread::Builder::new()
.name("smarm-io-epoll".into())
.spawn(move || epoll_loop(epollfd, epoll_comps, wake_write))?;
.spawn(move || epoll_loop(epollfd, epoll_waiters, rt))?;
Ok(Self {
tx,
completions,
wake_read,
wake_write,
waiters,
epollfd,
shutdown_read,
shutdown_write,
waiters: HashMap::new(),
pool_thread: Some(pool_thread),
epoll_thread: Some(epoll_thread),
outstanding: 0,
})
}
/// Hand a request to the pool. Increments `outstanding`.
/// Hand a request to the pool. The caller (scheduler.rs) increments
/// `io_outstanding` BEFORE calling — the pool decrements on completion,
/// and an increment that trailed the completion would underflow.
pub fn submit(&mut self, pid: Pid, epoch: u32, work: Box<dyn FnOnce() -> IoResult + Send>) {
self.outstanding += 1;
// Send can only fail if the pool has hung up, which only happens
// on shutdown. submit during shutdown is a bug.
if self.tx.send(Request { pid, epoch, work }).is_err() {
@@ -245,39 +236,13 @@ impl IoThread {
}
}
/// Drain every available completion. Caller (the scheduler) routes the
/// results and updates `outstanding` / `waiters` accordingly.
pub fn drain_completions(&mut self) -> Vec<Completion> {
let mut q = match self.completions.lock() {
Ok(g) => g,
Err(e) => panic!("smarm: io completions lock poisoned (core corrupt): {e}"),
};
let mut out = Vec::with_capacity(q.len());
while let Some(c) = q.pop_front() {
out.push(c);
}
out
}
pub fn wake_fd(&self) -> RawFd {
self.wake_read
}
/// Write the wake pipe directly: rouse every scheduler thread blocked in
/// its idle `poll_wake`. Used by the terminal (AllDone) path — an idle
/// sibling may be blocked on a snapshot that nothing will ever refresh
/// (an orphaned timer deadline, or `io_outstanding` from a waiter that
/// was stop-cancelled and so never produces a completion).
pub fn wake(&self) {
wake_scheduler(self.wake_write);
}
/// Register interest in `fd` becoming readable/writable; record `pid`
/// as the parked waiter. The epoll thread will push a `FdReady`
/// completion when the kernel signals.
/// as the parked waiter. The epoll thread unparks it on readiness.
/// The caller increments `io_fd_waiters` BEFORE calling (mirror of
/// `submit`'s contract) and decrements it again if this errors.
///
/// EPOLLONESHOT: one wakeup per registration. The scheduler must
/// `epoll_del` on completion to free the slot for re-registration.
/// EPOLLONESHOT: one wakeup per registration; the epoll thread DELs on
/// readiness, `cancel_waiter` DELs on an unwound wait.
pub fn epoll_register(
&mut self,
fd: RawFd,
@@ -286,20 +251,24 @@ impl IoThread {
readable: bool,
writable: bool,
) -> io::Result<()> {
let mut waiters = match self.waiters.lock() {
Ok(g) => g,
Err(e) => panic!("smarm: io waiters lock poisoned (core corrupt): {e}"),
};
// Two actors waiting on the same fd would be a misuse: the kernel
// delivers exactly one EPOLLONESHOT wakeup, so the second waiter
// would hang. Reject up front.
if self.waiters.contains_key(&fd) {
if waiters.contains_key(&fd) {
return Err(io::Error::new(
io::ErrorKind::AlreadyExists,
"fd already has a parked waiter",
));
}
// Belt-and-braces: the unwind guard in `wait_fd` is responsible for
// cleaning up a stopped waiter's registration, but a bare DEL is
// harmless if the fd isn't registered (ENOENT) and removes any leak
// a path we haven't thought of might leave behind.
// Belt-and-braces: `cancel_waiter` is responsible for cleaning up a
// stopped waiter's registration, but a bare DEL is harmless if the
// fd isn't registered (ENOENT) and removes any leak a path we
// haven't thought of might leave behind.
unsafe {
libc::epoll_ctl(self.epollfd, libc::EPOLL_CTL_DEL, fd, std::ptr::null_mut());
}
@@ -321,19 +290,29 @@ impl IoThread {
if r < 0 {
return Err(io::Error::last_os_error());
}
self.waiters.insert(fd, (pid, epoch));
waiters.insert(fd, (pid, epoch));
Ok(())
}
/// Remove `fd` from the epollfd. Called by the scheduler after a
/// `FdReady` completion, so the next `wait_readable(fd)` can ADD again.
///
/// Does NOT touch `waiters` — that's the scheduler's bookkeeping; this
/// is purely the kernel-side cleanup.
pub fn epoll_deregister(&mut self, fd: RawFd) {
// EPOLL_CTL_DEL of an already-removed fd returns ENOENT; ignore.
unsafe {
libc::epoll_ctl(self.epollfd, libc::EPOLL_CTL_DEL, fd, std::ptr::null_mut());
/// Remove `fd`'s waiter iff it is still `(pid, epoch)`, DELing the fd
/// from the epollfd in the same critical section. Returns whether the
/// entry was removed (the caller then decrements `io_fd_waiters`).
/// `false` means the epoll thread consumed the registration first —
/// the fd may already carry someone else's fresh ADD; hands off.
pub fn cancel_waiter(&mut self, fd: RawFd, pid: Pid, epoch: u32) -> bool {
let mut waiters = match self.waiters.lock() {
Ok(g) => g,
Err(e) => panic!("smarm: io waiters lock poisoned (core corrupt): {e}"),
};
if waiters.get(&fd) == Some(&(pid, epoch)) {
waiters.remove(&fd);
// EPOLL_CTL_DEL of an already-removed fd returns ENOENT; ignore.
unsafe {
libc::epoll_ctl(self.epollfd, libc::EPOLL_CTL_DEL, fd, std::ptr::null_mut());
}
true
} else {
false
}
}
}
@@ -354,7 +333,10 @@ impl Drop for IoThread {
let real_tx = std::mem::replace(&mut self.tx, dead_tx);
drop(real_tx);
// 3. Join both threads.
// 3. Join both threads. Safe even while the caller holds the
// runtime's `io` mutex: neither thread ever takes it (they reach
// the runtime through a Weak they upgrade per completion, and
// the epoll thread's only lock is `waiters`).
if let Some(h) = self.epoll_thread.take() {
let _ = h.join();
}
@@ -367,8 +349,6 @@ impl Drop for IoThread {
libc::close(self.epollfd);
libc::close(self.shutdown_read);
libc::close(self.shutdown_write);
libc::close(self.wake_read);
libc::close(self.wake_write);
}
}
}
@@ -379,36 +359,38 @@ impl Drop for IoThread {
const SHUTDOWN_EPOLL_TOKEN: u64 = u64::MAX;
// ---------------------------------------------------------------------------
// Pool loop
// Pool loop (producer: Blocking completions)
// ---------------------------------------------------------------------------
fn pool_loop(
rx: mpsc::Receiver<Request>,
completions: Arc<Mutex<VecDeque<Completion>>>,
wake_write: RawFd,
) {
fn pool_loop(rx: mpsc::Receiver<Request>, rt: Weak<RuntimeInner>) {
while let Ok(Request { pid, epoch, work }) = rx.recv() {
let result: IoResult = match panic::catch_unwind(panic::AssertUnwindSafe(work)) {
Ok(r) => r,
Err(payload) => Err(payload),
};
match completions.lock() {
Ok(mut g) => g.push_back(Completion::Blocking { pid, epoch, result }),
Err(e) => panic!("smarm: io completions lock poisoned (core corrupt): {e}"),
let Some(inner) = rt.upgrade() else { return };
// Stash the result under the cold lock (generation-checked: an
// actor stopped with the op in flight discards it), decrement the
// in-flight count, then wake through the epoch-matched unpark. The
// unpark's enqueue tail wakes a parked scheduler; the actor stays
// `live` until it resumes and finalizes, so the decrement's
// ordering against the termination verdict is not load-bearing.
if let Some(slot) = inner.slot_at(pid) {
let mut cold = slot.cold.lock();
if slot.generation() == pid.generation() {
cold.pending_io_result = Some(result);
}
}
wake_scheduler(wake_write);
inner.io_outstanding.fetch_sub(1, Ordering::AcqRel);
inner.unpark_at(pid, epoch);
}
}
// ---------------------------------------------------------------------------
// Epoll loop
// Epoll loop (producer: FdReady completions)
// ---------------------------------------------------------------------------
fn epoll_loop(
epollfd: RawFd,
completions: Arc<Mutex<VecDeque<Completion>>>,
wake_write: RawFd,
) {
fn epoll_loop(epollfd: RawFd, waiters: Waiters, rt: Weak<RuntimeInner>) {
// Buffer for epoll_wait. 64 is plenty for our scale; if a real load
// appears that needs more, this is a one-line change.
const MAX_EVENTS: usize = 64;
@@ -436,29 +418,41 @@ fn epoll_loop(
}
let mut shutdown_requested = false;
let mut pushed_any = false;
{
let mut q = match completions.lock() {
Ok(g) => g,
Err(e) => panic!("smarm: io completions lock poisoned (core corrupt): {e}"),
for ev in events.iter().take(n as usize) {
if ev.u64 == SHUTDOWN_EPOLL_TOKEN {
shutdown_requested = true;
continue;
}
let fd = ev.u64 as RawFd;
// Consume the registration: remove + DEL under the waiters
// lock (the ADD/DEL serialization — see module docs). A
// vanished entry means `cancel_waiter` beat us: the wake is
// already moot.
let entry = {
let mut w = match waiters.lock() {
Ok(g) => g,
Err(e) => {
panic!("smarm: io waiters lock poisoned (core corrupt): {e}")
}
};
let entry = w.remove(&fd);
if entry.is_some() {
unsafe {
libc::epoll_ctl(
epollfd,
libc::EPOLL_CTL_DEL,
fd,
std::ptr::null_mut(),
);
}
}
entry
};
for ev in events.iter().take(n as usize) {
if ev.u64 == SHUTDOWN_EPOLL_TOKEN {
shutdown_requested = true;
continue;
}
let fd = ev.u64 as RawFd;
let evs = ev.events;
q.push_back(Completion::FdReady {
fd,
events: evs,
});
pushed_any = true;
if let Some((pid, epoch)) = entry {
let Some(inner) = rt.upgrade() else { return };
inner.io_fd_waiters.fetch_sub(1, Ordering::AcqRel);
inner.unpark_at(pid, epoch);
}
}
if pushed_any {
wake_scheduler(wake_write);
}
if shutdown_requested {
return;
@@ -466,27 +460,8 @@ fn epoll_loop(
}
}
/// Write one byte to the scheduler's wake pipe. Retries on EINTR; ignores
/// EAGAIN (pipe full means there's already an outstanding wake we haven't
/// consumed yet, which is sufficient).
fn wake_scheduler(wake_write: RawFd) {
let buf: [u8; 1] = [0];
unsafe {
loop {
let n = libc::write(wake_write, buf.as_ptr() as *const _, 1);
if n < 0 {
let e = *libc::__errno_location();
if e == libc::EINTR {
continue;
}
}
break;
}
}
}
// ---------------------------------------------------------------------------
// Pipe helpers (unchanged from v0.2)
// Pipe helper
// ---------------------------------------------------------------------------
fn make_pipe() -> io::Result<(RawFd, RawFd)> {
@@ -497,50 +472,3 @@ fn make_pipe() -> io::Result<(RawFd, RawFd)> {
}
Ok((fds[0], fds[1]))
}
/// Drain pending bytes from the wake pipe. Nonblocking (pipe is O_NONBLOCK).
///
/// DISCIPLINE: called only by the phase-1 drain-lock winner, immediately
/// before `drain_completions`. Bytes are the notification channel for
/// completions; consuming one anywhere else can strand the completion it
/// announces (see the lost-wakeup note at the call site in `schedule_loop`).
pub fn drain_wake_pipe(fd: RawFd) {
let mut buf = [0u8; 64];
loop {
let n = unsafe { libc::read(fd, buf.as_mut_ptr() as *mut _, buf.len()) };
if n <= 0 {
break;
}
}
}
/// Block on `fd` for up to `timeout`, returning when either there's data
/// to read or the timeout elapses. `None` for `timeout` means wait forever.
pub fn poll_wake(fd: RawFd, timeout: Option<std::time::Duration>) {
let timeout_ms: libc::c_int = match timeout {
None => -1,
Some(d) => {
let ms = d.as_millis();
if ms > i32::MAX as u128 {
i32::MAX
} else {
ms as i32
}
}
};
let mut pfd = libc::pollfd {
fd,
events: libc::POLLIN,
revents: 0,
};
loop {
let r = unsafe { libc::poll(&mut pfd as *mut _, 1, timeout_ms) };
if r < 0 {
let e = unsafe { *libc::__errno_location() };
if e == libc::EINTR {
continue;
}
}
break;
}
}
+6 -3
View File
@@ -12,6 +12,7 @@
//! See `LOOM.md` for the design intent and the deferred-for-later list.
pub mod stack;
pub(crate) mod signal;
pub mod context;
pub mod preempt;
pub mod pid;
@@ -32,6 +33,7 @@ pub mod introspect;
#[cfg(feature = "observer")]
pub mod observer;
pub mod runtime;
pub(crate) mod park;
pub(crate) mod raw_mutex;
pub(crate) mod slot_state;
pub(crate) mod sync_shim;
@@ -63,7 +65,7 @@ pub use gen_statem::{
CallError as GenStatemCallError, Cx, Machine, Reply, Resolution, SendError as GenStatemSendError,
GenStatemRef,
};
pub use introspect::{
pub use introspect::{StackInfo,
actor_info, snapshot, tree, tree_from, ActorInfo, ActorState, RuntimeSnapshot, RuntimeTree,
TreeNode, SNAPSHOT_FORMAT_VERSION,
};
@@ -82,8 +84,9 @@ pub use runtime::{init, Config, Runtime};
pub use scheduler::{
block_on_io, cancel_timer, request_stop, run, self_pid, send_after, send_after_named,
send_after_named_wall, send_after_wall, sleep, sleep_wall,
spawn, spawn_addr, spawn_under, wait_readable, wait_readable_timeout, wait_writable,
wait_writable_timeout, yield_now, FdArm, JoinError, JoinHandle,
spawn, spawn_addr, spawn_addr_with, spawn_under, spawn_under_with, spawn_with,
wait_readable, wait_readable_timeout, wait_writable,
wait_writable_timeout, yield_now, FdArm, JoinError, JoinHandle, SpawnOpts,
};
pub use supervisor::{ChildSpec, OneForOne, Restart, Signal, Strategy};
pub use timer::TimerId;
+102 -62
View File
@@ -1,49 +1,85 @@
//! Process monitors.
//! Find out when another actor dies, without it knowing or caring that you're
//! watching.
//!
//! `monitor(target)` asks the runtime to deliver a single [`Down`] when
//! `target` terminates, and hands back a [`Monitor`] — the [`Receiver`] to read
//! it from, plus the identity (`id`, `target`) needed to take the registration
//! back down with [`demonitor`]. A monitor is:
//! Say one actor manages a pool of workers and needs to know when a worker
//! exits, so it can replace it. The worker does not need to know it is being
//! watched, and nothing about the worker's own behavior should change because
//! someone is watching it. That is what [`monitor`] is for: call
//! `monitor(target)` to get a [`Monitor`], and read exactly one [`Down`]
//! message off `monitor.rx` whenever `target` terminates, however it
//! terminates.
//!
//! - **unidirectional** — the watcher learns of the target's death, but the
//! target learns nothing of the watcher, and the watcher is unaffected by
//! the death beyond the notification (contrast a *link*, which propagates
//! failure);
//! - **one-shot** — exactly one `Down` is ever sent for a given monitor.
//! The returned channel closes afterwards, so a second `recv()` yields
//! `Err(RecvError)`.
//! ```
//! use smarm::{monitor, run, spawn, DownReason};
//!
//! This generalizes the older single-`supervisor_channel` mechanism: a
//! supervisor is just a hard-wired monitor that the parent installs at spawn
//! time. Here any actor may monitor any pid, any number of times.
//! run(|| {
//! let worker = spawn(|| {
//! // does some work, then returns
//! });
//! let pid = worker.pid();
//!
//! ## Reasons
//! let m = monitor(pid);
//! let _ = worker.join();
//!
//! [`DownReason`] is deliberately payload-free. A panicking actor's payload
//! has a single owner and is delivered to whoever `join()`s the actor (as
//! `JoinError`); a monitor only learns *that* it panicked, not the value.
//! Monitoring a pid that is already gone (reclaimed, or never alive) yields
//! [`DownReason::NoProc`] immediately, mirroring Erlang's `noproc`.
//! let down = m.rx.recv().expect("monitor channel closed before Down");
//! assert_eq!(down.pid, pid);
//! assert_eq!(down.reason, DownReason::Exit);
//! });
//! ```
//!
//! ## Demonitoring
//! A monitor is one-directional and one-shot:
//!
//! Each `monitor()` registration is tagged with a process-unique [`MonitorId`].
//! [`demonitor`] removes the registration named by a [`Monitor`] from its
//! target's slot, returning `Some(id)` if a live registration was found or
//! `None` if it had already fired (or the target is gone). Dropping the
//! [`Monitor`] afterwards discards any `Down` that the target had *already*
//! queued — the equivalent of Erlang's `demonitor(Ref, [flush])`.
//! - **One-directional**: the watcher learns that the target died, but the
//! target is completely unaffected. It never learns it was being watched,
//! and its own behavior and lifetime do not change because of the monitor.
//! This is the opposite of a [`link`](mod@crate::link), which is bidirectional:
//! linking two actors means an abnormal death on either side can bring the
//! other down too. Reach for a monitor when you just want to *know*; reach
//! for a link when a peer's crash should actually stop you.
//! - **One-shot**: you get exactly one [`Down`] per `monitor()` call, then the
//! channel closes. Calling `monitor` again on the same target (or a
//! different one) gives you an independent registration with its own
//! [`Monitor`] and its own one-shot channel; nothing stops you from
//! monitoring the same actor many times over; each call is watched and
//! fires on its own.
//!
//! ## Races
//! ## Why a monitor never hands you the panic value
//!
//! Registration (below) and `finalize_actor` (in `runtime`) both run under the
//! shared-state mutex, so a target that is still alive when its monitor is
//! registered is guaranteed to deliver a real `Down`; there is no window in
//! which the death slips between the liveness check and the registration.
//! `demonitor` is protected by the generation half of the pid: if the target
//! has died and its slot index been recycled, `slot_mut(target)` fails the
//! generation check and `demonitor` is a clean no-op — it can never strip a
//! *different* actor's monitor that happens to share the slot index.
//! If the target panicked, [`Down`] tells you *that* it panicked
//! ([`DownReason::Panic`]), but not the panic's payload. The payload has a
//! single owner: it is handed to whichever caller `join()`s the actor's
//! [`JoinHandle`](crate::JoinHandle), as a `JoinError`. A monitor only needs
//! to know that something went wrong, not reproduce the exact value that
//! caused it, so it gets the reason and nothing else.
//!
//! Monitoring a target that is already gone (it finished and was cleaned up,
//! or the pid never pointed at a real actor) is not an error: you get a
//! [`Down`] with [`DownReason::NoProc`] right away, instead of waiting
//! forever for something that already happened.
//!
//! ## Stopping a monitor early
//!
//! [`demonitor`] cancels a monitor before it fires. If the registration was
//! still live, it removes it and returns `Some` of the monitor's id: no
//! `Down` will arrive on that channel from here on. If the target had already
//! died and its `Down` already sent, there is nothing left to cancel and
//! `demonitor` returns `None`; the `Down` you already have (or that is
//! already sitting in the channel) is unaffected.
//!
//! If you want to cancel *and* make sure a `Down` that already arrived is
//! discarded without reading it, just drop the [`Monitor`]: dropping it closes
//! its receiver, and any queued `Down` is dropped along with it.
//!
//! ## Correctness notes for implementers
//!
//! A target that is still alive at the moment `monitor()` registers is
//! guaranteed to eventually produce a real `Down`: registration and the
//! target's own termination bookkeeping run under the same lock, so there is
//! no window in which the target could die without the just-added
//! registration seeing it. `demonitor` is similarly race-free against a target
//! that has since died and had its slot reused by a new, unrelated actor: it
//! is checked against the exact monitored incarnation, so it can never remove
//! a different actor's registration by accident, it simply reports `None`.
use crate::channel::{channel, Receiver, Sender};
use crate::pid::Pid;
@@ -51,8 +87,8 @@ use crate::scheduler::with_runtime;
/// Why a monitored actor went down.
///
/// `Copy` because it carries no payload — see the module docs for why the
/// panic payload is *not* included here.
/// Carries no payload: see the module docs for why a monitor never receives
/// the panic value itself.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum DownReason {
/// The target returned normally.
@@ -76,21 +112,22 @@ pub struct Down {
pub reason: DownReason,
}
/// A process-unique identifier for one `monitor()` registration.
/// A unique identifier for one [`monitor`] registration.
///
/// Opaque and `Copy`. Allocated from a monotonic counter in shared state, so
/// it is never reused for the lifetime of the runtime — distinct `monitor()`
/// calls on the same target get distinct ids, which is what lets [`demonitor`]
/// tear down exactly one of several monitors on a target.
/// Opaque and `Copy`. Never reused for the life of the runtime, so if you
/// monitor the same target more than once, each call's id is distinct. This
/// is what lets [`demonitor`] tear down exactly one of several monitors on
/// the same target without disturbing the others.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub struct MonitorId(pub(crate) u64);
/// A live monitor: the receiving end of the one-shot [`Down`] channel, plus the
/// identity needed to [`demonitor`] it.
///
/// Read the notification from [`Monitor::rx`]. Not `Clone` (the receiver is a
/// single consumer). Dropping it closes the receiving end; if a `Down` was
/// already queued it is discarded with the channel.
/// Read the notification from [`Monitor::rx`]. Not `Clone`, since only one
/// side is meant to consume it. Dropping a `Monitor` closes the receiving
/// end; if a `Down` had already arrived but was never read, it is discarded
/// along with it.
pub struct Monitor {
/// This registration's process-unique id.
pub id: MonitorId,
@@ -110,10 +147,11 @@ pub fn monitor<A>(target: Pid<A>) -> Monitor {
let target = target.erase();
let (tx, rx) = channel::<Down>();
// Register under the target's cold lock. `tx.clone()` takes the channel's
// own lock — a Channel-class RawMutex, explicitly permitted *under* a Leaf
// (cold) lock by the lock order (see raw_mutex.rs). We must still not
// *send* under the lock, as `Sender::send` can unpark a parked receiver,
// Implementation note: registration happens under the target's cold
// lock. `tx.clone()` takes the channel's own lock, a Channel-class
// RawMutex, which is explicitly permitted under a Leaf (cold) lock by
// the lock order documented in raw_mutex.rs. We must still not *send*
// under the lock, since `Sender::send` can unpark a parked receiver,
// and there's no reason to nest that.
let (id, registered) = with_runtime(|inner| {
let id = inner.alloc_monitor_id();
@@ -140,19 +178,21 @@ pub fn monitor<A>(target: Pid<A>) -> Monitor {
}
/// Cancel the monitor `m`. Returns `Some(id)` if a live registration was found
/// on the target's slot and removed, or `None` if there was nothing to remove
/// — the target already fired its `Down` (the registration is drained on
/// finalize), was never alive (`NoProc`), or has been reclaimed.
/// and removed, so no `Down` will arrive on `m.rx` from here on. Returns
/// `None` if there was nothing left to remove: the target had already gone
/// down and its `Down` was already sent (or is already sitting in the
/// channel, unread).
///
/// This stops any *future* `Down`. To also discard a `Down` the target may have
/// *already* queued (the finalize-races-demonitor case), drop `m` afterwards;
/// dropping the [`Monitor`] closes its receiver and the queued notice goes with
/// it — the analogue of Erlang's `demonitor(Ref, [flush])`.
/// This only stops a *future* `Down`. If you also want to discard a `Down`
/// that already arrived (or is about to, in a race with this call), drop `m`
/// instead of, or in addition to, calling this: dropping the [`Monitor`]
/// closes its receiver and any queued notice is discarded with it.
pub fn demonitor(m: &Monitor) -> Option<MonitorId> {
// Remove the registration under the target's cold lock, but move the
// `Sender` *out* and let it drop only after the lock is released:
// dropping the last sender runs `Sender::drop`, which may unpark a parked
// receiver — legal under a cold lock, but pointless to nest.
// Implementation note: the registration is removed under the target's
// cold lock, but the `Sender` is moved *out* and dropped only after the
// lock is released. Dropping the last sender runs `Sender::drop`, which
// may unpark a parked receiver; legal under a cold lock, but pointless
// to nest.
let removed: Option<(MonitorId, Sender<Down>)> = with_runtime(|inner| {
let slot = inner.slot_at(m.target)?;
let mut cold = slot.cold.lock();
+120 -9
View File
@@ -1,12 +1,89 @@
//! Actor-aware mutex with mandatory timeout.
//! Shared mutable state across actors, when a channel is overkill.
//!
//! `Mutex<T>` parks the calling *green* thread on contention rather than
//! blocking the OS thread. Every lock attempt is bounded by a timeout.
//! smarm actors normally coordinate by sending messages, and for a piece of
//! owned state the right tool is usually a `gen_server`: one actor holds the
//! data and everyone else talks to it. Sometimes that is more machinery than
//! you need, and plain shared, lockable state is simpler: [`Mutex<T>`] is
//! that escape hatch. It behaves like `std::sync::Mutex<T>`, guarding a value
//! of type `T` behind a guard that gives you `&mut T` while held, but it is
//! built for smarm's actors rather than OS threads.
//!
//! Internals use `Arc<std::sync::Mutex<...>>` so the type is genuinely
//! `Send + Sync` and can be shared across scheduler threads.
//! The key difference from `std::sync::Mutex` is what happens on contention.
//! [`Mutex::lock`] parks the calling actor (a cooperatively scheduled green
//! thread) rather than blocking the underlying OS thread, so other actors on
//! the same OS thread keep running while it waits. And every lock attempt is
//! bounded by a timeout: an actor that hangs on to the lock forever (stuck in
//! a bug, or just slow) would otherwise wedge every other actor waiting on
//! it, so smarm makes the wait bounded by default instead of leaving it up
//! to you to remember.
//!
//! Fairness: FIFO. Poisoning: none. Reentrance: deadlock (caller bug).
//! ## A first lock
//!
//! ```
//! use smarm::{run, spawn, Mutex};
//!
//! run(|| {
//! let counter = Mutex::new(0u32);
//!
//! // Mutex::clone() is cheap and hands out another handle to the SAME
//! // underlying value, much like Arc::clone: every clone shares one lock
//! // and one value, so mutations through one are visible through all.
//! let a = counter.clone();
//! let b = counter.clone();
//!
//! let h1 = spawn(move || {
//! let mut guard = a.lock().unwrap();
//! *guard += 1;
//! });
//! let h2 = spawn(move || {
//! let mut guard = b.lock().unwrap();
//! *guard += 1;
//! });
//! h1.join().unwrap();
//! h2.join().unwrap();
//!
//! assert_eq!(*counter.lock().unwrap(), 2);
//! });
//! ```
//!
//! ## Choosing a timeout
//!
//! [`Mutex::lock`] waits up to [`DEFAULT_TIMEOUT`] (30 seconds) before giving
//! up with [`LockTimeout`]. To use a different bound for one call, use
//! [`Mutex::lock_timeout`] instead; to change the default for every future
//! `lock()` call on this mutex (including through its clones), use
//! [`Mutex::set_default_timeout`]. If you never want to wait at all, use
//! [`Mutex::try_lock`], which returns immediately whether or not the lock was
//! free.
//!
//! ## Fairness and panics
//!
//! Waiters are granted the lock in the order they started waiting (FIFO), so
//! no actor can be starved by later arrivals repeatedly cutting in line.
//!
//! This mutex never poisons. `std::sync::Mutex` marks itself poisoned if a
//! thread panics while holding the lock, because a partly mutated value might
//! be left behind for the next lock holder to see. smarm's actors already
//! rely on `Drop` running during unwinding to release the lock, so if a
//! holder panics, [`MutexGuard::drop`] still runs and the next waiter is
//! granted the lock normally. It is the same tradeoff `std::sync::Mutex`
//! offers you if you choose to ignore poisoning: you may see a value left
//! mid-update by the panicking actor, so a panic inside a critical section is
//! still a bug worth fixing, just not one that also wedges every future lock
//! attempt.
//!
//! Locking a mutex you already hold (on the same actor) does not queue
//! behind yourself: it deadlocks, the same way relocking a non-reentrant
//! `std::sync::Mutex` does. Don't call `lock` while already holding a guard
//! from the same `Mutex`.
//!
//! ## Outside the runtime
//!
//! `Mutex<T>` also works when called from plain code that is not running as
//! a smarm actor (for example, in a test's setup code before calling
//! [`run`](crate::run)). There, an actor's cooperative park has no meaning,
//! so a lock attempt instead blocks the calling OS thread directly until the
//! mutex is free; there is no timeout on this path.
use crate::pid::Pid;
use crate::scheduler;
@@ -15,8 +92,14 @@ use std::collections::VecDeque;
use std::sync::{Arc, Mutex as StdMutex};
use std::time::Duration;
/// How long [`Mutex::lock`] waits for the lock before giving up, unless
/// overridden per-mutex with [`Mutex::set_default_timeout`] or per-call with
/// [`Mutex::lock_timeout`].
pub const DEFAULT_TIMEOUT: Duration = Duration::from_secs(30);
/// Returned by [`Mutex::lock`] / [`Mutex::lock_timeout`] when the timeout
/// elapses before the lock became available. The lock attempt is abandoned;
/// nothing was acquired, and the mutex's value is unaffected.
#[derive(Debug, PartialEq, Eq, Clone, Copy)]
pub struct LockTimeout;
@@ -70,7 +153,7 @@ impl TimerTarget for MutexCore {
};
// Remove from waiters only if still there with matching epoch.
// If the lock was already granted (holder == Some(pid)), the
// timer fired after the grant — treat as no-op; the actor
// timer fired after the grant: treat as no-op; the actor
// will see `is_holder == true` and return Ok.
if st.holder == Some(pid) {
return;
@@ -100,6 +183,8 @@ pub struct Mutex<T> {
}
impl<T> Mutex<T> {
/// Wrap `value` in a new mutex, initially unlocked, with the default
/// lock timeout ([`DEFAULT_TIMEOUT`]).
pub fn new(value: T) -> Self {
Self {
core: Arc::new(MutexCore::new(DEFAULT_TIMEOUT)),
@@ -107,6 +192,11 @@ impl<T> Mutex<T> {
}
}
/// Change how long future [`lock`](Self::lock) calls on this mutex wait
/// before giving up. Applies to every clone of this `Mutex` (they share
/// one underlying lock), and to `lock` calls already in progress that
/// have not yet started waiting. Does not affect [`lock_timeout`](Self::lock_timeout)
/// calls, which always use the timeout passed in.
pub fn set_default_timeout(&self, timeout: Duration) {
match self.core.state.lock() {
Ok(mut st) => st.default_timeout = timeout,
@@ -114,6 +204,12 @@ impl<T> Mutex<T> {
}
}
/// Acquire the lock, waiting up to this mutex's default timeout
/// ([`DEFAULT_TIMEOUT`], or whatever [`set_default_timeout`](Self::set_default_timeout)
/// last set) if it is currently held elsewhere. Returns a [`MutexGuard`]
/// that releases the lock when dropped, or [`LockTimeout`] if the
/// deadline passes first. To use a one-off timeout instead of the
/// mutex's default, call [`lock_timeout`](Self::lock_timeout) directly.
pub fn lock(&self) -> Result<MutexGuard<'_, T>, LockTimeout> {
let timeout = match self.core.state.lock() {
Ok(st) => st.default_timeout,
@@ -122,6 +218,10 @@ impl<T> Mutex<T> {
self.lock_timeout(timeout)
}
/// Acquire the lock, waiting up to `timeout` (ignoring this mutex's
/// default) if it is currently held elsewhere. Returns a [`MutexGuard`]
/// that releases the lock when dropped, or [`LockTimeout`] if `timeout`
/// elapses first with the lock still unavailable.
pub fn lock_timeout(&self, timeout: Duration) -> Result<MutexGuard<'_, T>, LockTimeout> {
// Outside the runtime (e.g. in tests, after run() returns) there is no
// current actor PID. Fall back to a blocking std::sync::Mutex acquire.
@@ -157,7 +257,7 @@ impl<T> Mutex<T> {
Ok(g) => g,
Err(e) => panic!("smarm: mutex state lock poisoned (core corrupt): {e}"),
};
// begin_wait is lock-free — legal under the state lock; this
// begin_wait is lock-free (legal under the state lock); this
// makes the epoch atomic with the registration's visibility to
// grants and timeouts.
let epoch = scheduler::begin_wait();
@@ -170,7 +270,7 @@ impl<T> Mutex<T> {
scheduler::insert_wait_timer(deadline, me, target, epoch);
scheduler::park_current();
// Resumed — precisely: only our grant or our timer can wake this
// Resumed, precisely: only our grant or our timer can wake this
// wait (both epoch-stamped; a stop wake unwinds out of
// park_current). The one-shot interpretation below is therefore
// exhaustive. Are we the holder?
@@ -193,6 +293,9 @@ impl<T> Mutex<T> {
}
}
/// Acquire the lock only if it is immediately available: never parks and
/// never waits. Returns `Some` with a [`MutexGuard`] if the lock was
/// free, `None` if it is currently held elsewhere.
pub fn try_lock(&self) -> Option<MutexGuard<'_, T>> {
let me = crate::actor::current_pid()?;
let mut st = match self.core.state.lock() {
@@ -234,6 +337,10 @@ impl<T> Mutex<T> {
}
impl<T> Clone for Mutex<T> {
/// Cheap: hands back another handle to the same underlying lock and
/// value, the way `Arc::clone` does. All clones of a `Mutex` share one
/// lock and one protected value; locking through any clone excludes
/// every other clone.
fn clone(&self) -> Self {
Self { core: self.core.clone(), value: self.value.clone() }
}
@@ -247,6 +354,10 @@ unsafe impl<T: Send> Sync for Mutex<T> {}
// Guard
// ---------------------------------------------------------------------------
/// Grants access to the value inside a [`Mutex`] while the lock is held.
/// Dereferences to `&T` and `&mut T`. Dropping the guard releases the lock
/// and, if another actor is waiting, wakes the next one in arrival order.
/// Returned by [`Mutex::lock`], [`Mutex::lock_timeout`], and [`Mutex::try_lock`].
pub struct MutexGuard<'a, T> {
mutex: &'a Mutex<T>,
value: Option<T>,
+994
View File
@@ -0,0 +1,994 @@
//! Scheduler park/wake coordination layer (RFC 018).
//!
//! Schedulers never touch an fd to sleep: they park on a per-thread
//! [`Parker`] and are woken through an idle-mask protocol the runtime owns
//! outright. IO backends (epoll today, io_uring later) are *producers*
//! behind a two-call contract — make actors runnable, then wake — which is
//! what makes backend selection tractable (RFC 018 §step-back).
//!
//! Three pieces, all in [`Coordinator`]:
//!
//! - **Parker** (one per scheduler): permit semantics, `std::thread::park`
//! shaped — an unpark delivered before the park sets a permit; the next
//! park consumes it and returns immediately. This single property closes
//! the check-then-park race. Linux: `futex(2)` `FUTEX_WAIT`/`FUTEX_WAKE`
//! (private) with a nanosecond-precision relative `timespec` — the
//! `as_millis` truncation defect of the retired wake pipe is
//! unrepresentable here. Loom / non-Linux: `Mutex<bool>` + `Condvar`
//! (the loom models run against this build).
//! - **Idle mask**: an `AtomicU64` bitmask of parked scheduler ids
//! (construction asserts ≤ 64 schedulers). Park protocol: set own bit,
//! run the caller's mandatory post-publish re-check, then wait. A
//! producer that published work before observing our bit has left us
//! work the re-check finds; one that observes the bit wakes us.
//!
//! The publish/re-check pair is a store-buffer (Dekker) shape. Two sound
//! resolutions coexist here, chosen per call-site cost profile: the
//! **fence handshake** on the hot producer path
//! ([`Coordinator::wake_one_if_idle`]: publish work; `fence(SeqCst)`;
//! one *Relaxed* mask load — pairing with the consumer's `fetch_or(bit)`;
//! `fence(SeqCst)`; re-check inside [`Coordinator::park`]), so the
//! pure-compute hot path (everyone busy, mask 0) never takes the shared
//! mask line exclusive — the RFC's "one relaxed load" fast path, made
//! sound; and the **same-location-RMW read** (`fetch_or(0)`, in
//! [`Coordinator::wake_one`] / [`Coordinator::idle_mask`]) for the rare
//! paths (chain rule — at most one per wake) where reading the latest
//! mask by modification-order coherence is worth an RMW. The loom models
//! drive the fence pattern end to end (a fence-less plain-load draft
//! would — and did — fail model 1/2 with a lost wake, as it must).
//! - **Timekeeper**: at most one parked scheduler holds the timer
//! deadline (RFC 018 §timers) so a timer expiry wakes one scheduler,
//! not a herd. The role is an atomic `(holder id, armed deadline)`
//! pair; a timer insertion with an earlier deadline wakes the holder to
//! re-peek. Arm / insert-check MUST be serialized by the caller (the
//! timers mutex in the runtime) — the atomics exist so the *busy-path
//! due-check* (one Relaxed load, [`Coordinator::armed_deadline_nanos`])
//! and the wake stay lock-free. All races are biased over-wake: a
//! spurious permit costs one failed pop; a missed wake would cost a
//! stranded actor, and is unrepresentable under the serialization rule.
//!
//! Every wake here is *at most one* futex round-trip and wakes *exactly
//! one* scheduler by construction (`wake_one` CASes a bit clear before
//! unparking its owner) — there is no shared level-triggered anything
//! left to herd on.
//!
//! Standalone until the runtime swap (RFC 018 commit 2): nothing outside
//! tests constructs a [`Coordinator`] yet.
use crate::sync_shim::{fence, AtomicU64, Ordering};
use std::time::Instant;
/// Sentinel for "no timekeeper" in the holder word.
const NO_TIMEKEEPER: u64 = u64::MAX;
/// Sentinel for "no armed deadline" in the deadline word. Also what the
/// busy-path due-check compares against: `now_nanos < armed` is one branch.
pub(crate) const NO_DEADLINE: u64 = u64::MAX;
/// Outcome of a [`Coordinator::park`] call.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(crate) enum ParkResult {
/// A permit was consumed (wake delivered before or during the park).
Woken,
/// The deadline passed with no wake. Only possible when a deadline was
/// supplied (and never under loom, which has no time — see `park`).
TimedOut,
/// The post-publish re-check found work; the thread never blocked.
/// A permit may still be pending (a racing `wake_one` picked us after
/// the bit was set) — it will surface as one spurious `Woken` on a
/// later park. Benign: over-wake by design.
WorkFound,
}
// ---------------------------------------------------------------------------
// Parker — permit-semantics thread parking
// ---------------------------------------------------------------------------
/// Linux, non-loom: futex on a state word.
///
/// States: EMPTY (no permit, nobody waiting), PARKED (a thread is, or is
/// about to be, in `futex_wait`), NOTIFIED (permit pending). The classic
/// std-parker protocol: `unpark` swaps to NOTIFIED and futex-wakes iff it
/// displaced PARKED; `park` CASes EMPTY→PARKED, waits, and consumes
/// NOTIFIED on every exit path.
#[cfg(all(target_os = "linux", not(loom)))]
mod parker {
use std::sync::atomic::{AtomicU32, Ordering};
use std::time::Instant;
const EMPTY: u32 = 0;
const PARKED: u32 = 1;
const NOTIFIED: u32 = 2;
pub(super) struct Parker {
state: AtomicU32,
}
impl Parker {
pub(super) fn new() -> Self {
Self { state: AtomicU32::new(EMPTY) }
}
/// Returns `true` = woken (permit consumed), `false` = timed out.
pub(super) fn park(&self, deadline: Option<Instant>) -> bool {
// Fast path: consume a pending permit without blocking.
if self
.state
.compare_exchange(EMPTY, PARKED, Ordering::AcqRel, Ordering::Acquire)
.is_err()
{
// Only NOTIFIED can be here (one thread parks at a time).
self.state.store(EMPTY, Ordering::Release);
return true;
}
loop {
let timeout = match deadline {
None => None,
Some(d) => {
let now = Instant::now();
if d <= now {
// Deadline passed: cancel the park. The swap
// races a concurrent unpark — if it delivered
// NOTIFIED first, report Woken (never lose a
// permit).
return self.state.swap(EMPTY, Ordering::AcqRel) == NOTIFIED;
}
Some(d - now)
}
};
futex_wait(&self.state, PARKED, timeout);
if self
.state
.compare_exchange(NOTIFIED, EMPTY, Ordering::AcqRel, Ordering::Acquire)
.is_ok()
{
return true;
}
// Spurious wake or timeout with state still PARKED: loop —
// the deadline check at the top decides.
}
}
pub(super) fn unpark(&self) {
if self.state.swap(NOTIFIED, Ordering::AcqRel) == PARKED {
futex_wake(&self.state, 1);
}
}
}
/// `FUTEX_WAIT` with a *relative* nanosecond timeout (`CLOCK_MONOTONIC`
/// per futex(2) for relative waits). No millisecond conversion anywhere:
/// the timespec carries the full sub-ms remainder (RFC 018 kills the
/// `as_millis` truncation structurally).
fn futex_wait(word: &AtomicU32, expected: u32, timeout: Option<std::time::Duration>) {
let ts;
let ts_ptr: *const libc::timespec = match timeout {
Some(d) => {
ts = libc::timespec {
tv_sec: d.as_secs() as libc::time_t,
tv_nsec: d.subsec_nanos() as libc::c_long,
};
&ts
}
None => std::ptr::null(),
};
// Errors (EAGAIN: word changed; ETIMEDOUT; EINTR) all mean "return
// and let the caller's state machine decide" — deliberately ignored.
unsafe {
libc::syscall(
libc::SYS_futex,
word.as_ptr(),
libc::FUTEX_WAIT | libc::FUTEX_PRIVATE_FLAG,
expected,
ts_ptr,
);
}
}
fn futex_wake(word: &AtomicU32, n: u32) {
unsafe {
libc::syscall(
libc::SYS_futex,
word.as_ptr(),
libc::FUTEX_WAKE | libc::FUTEX_PRIVATE_FLAG,
n,
);
}
}
}
/// Loom / non-Linux: `Mutex<bool>` permit + `Condvar` — loom's own model
/// of a parker, and the portable fallback. Under loom the deadline is
/// ignored (loom has no time); the models exercise wake paths only.
#[cfg(any(loom, not(target_os = "linux")))]
mod parker {
use crate::sync_shim::{Condvar, Mutex};
use std::time::Instant;
pub(super) struct Parker {
permit: Mutex<bool>,
cv: Condvar,
}
impl Parker {
pub(super) fn new() -> Self {
Self { permit: Mutex::new(false), cv: Condvar::new() }
}
/// Returns `true` = woken (permit consumed), `false` = timed out.
pub(super) fn park(&self, deadline: Option<Instant>) -> bool {
let mut permit = match self.permit.lock() {
Ok(g) => g,
Err(_) => panic!("smarm: parker permit lock poisoned (core corrupt)"),
};
loop {
if *permit {
*permit = false;
return true;
}
#[cfg(loom)]
{
// Loom has no clock: block until a wake. Models must
// deliver one (a park nobody wakes is a real deadlock
// and loom reports it as such).
let _ = deadline;
permit = match self.cv.wait(permit) {
Ok(g) => g,
Err(_) => panic!("smarm: parker cv poisoned (core corrupt)"),
};
}
#[cfg(not(loom))]
{
match deadline {
None => {
permit = match self.cv.wait(permit) {
Ok(g) => g,
Err(_) => panic!("smarm: parker cv poisoned (core corrupt)"),
};
}
Some(d) => {
let now = Instant::now();
if d <= now {
return false;
}
permit = match self.cv.wait_timeout(permit, d - now) {
Ok((g, _)) => g,
Err(_) => panic!("smarm: parker cv poisoned (core corrupt)"),
};
}
}
}
}
}
pub(super) fn unpark(&self) {
let mut permit = match self.permit.lock() {
Ok(g) => g,
Err(_) => panic!("smarm: parker permit lock poisoned (core corrupt)"),
};
*permit = true;
self.cv.notify_one();
}
}
}
use parker::Parker;
// ---------------------------------------------------------------------------
// Coordinator — idle mask + wake protocol + timekeeper
// ---------------------------------------------------------------------------
pub(crate) struct Coordinator {
parkers: Box<[Parker]>,
/// Bit `i` set = scheduler `i` is parked or committed to parking (set
/// before the re-check; cleared by `wake_one`'s CAS or by the parker
/// itself on return). AcqRel same-location-RMW handshake — see module
/// docs (no SeqCst needed: every producer-side read is an RMW).
idle: AtomicU64,
/// Timekeeper holder id, or `NO_TIMEKEEPER`. Written under the
/// caller's timer serialization (arm/disarm/insert-check); read
/// lock-free by the insert wake path.
tk_holder: AtomicU64,
/// Armed deadline as nanos since `origin`, or `NO_DEADLINE`. Written
/// only by the timekeeper arm/disarm protocol.
tk_armed: AtomicU64,
/// Earliest KNOWN timer deadline (nanos since `origin`), or
/// `NO_DEADLINE` — independent of whether any scheduler is parked,
/// which is what the timekeeper's `tk_armed` cannot give: under
/// saturation nobody parks and nobody arms, yet due timers must still
/// fire (ratified design point (a)). Maintained under the caller's
/// timers mutex (`note_deadline` on insert, `refresh_deadline` after a
/// pop/peek); read lock-free by the busy-path due-check.
next_deadline: AtomicU64,
/// Time origin for the nanos encoding.
origin: Instant,
}
impl Coordinator {
pub(crate) fn new(schedulers: usize) -> Self {
assert!(
(1..=64).contains(&schedulers),
"smarm: scheduler count must be 1..=64 (idle mask is one u64); got {schedulers}"
);
Self {
parkers: (0..schedulers).map(|_| Parker::new()).collect(),
idle: AtomicU64::new(0),
tk_holder: AtomicU64::new(NO_TIMEKEEPER),
tk_armed: AtomicU64::new(NO_DEADLINE),
next_deadline: AtomicU64::new(NO_DEADLINE),
origin: Instant::now(),
}
}
/// Encode a deadline for the armed snapshot / busy-path compare.
/// Saturating: a deadline at-or-before `origin` encodes as 0 (always
/// due), one beyond ~584 years as `NO_DEADLINE - 1`.
pub(crate) fn deadline_nanos(&self, deadline: Instant) -> u64 {
let nanos = deadline
.checked_duration_since(self.origin)
.map(|d| d.as_nanos())
.unwrap_or(0);
if nanos >= NO_DEADLINE as u128 {
NO_DEADLINE - 1
} else {
nanos as u64
}
}
/// Park scheduler `id` until a wake, the deadline, or a positive
/// re-check. Protocol: (1) publish own idle bit (SeqCst), (2) run
/// `recheck` — it MUST re-read the work source with an ordering that
/// pairs with the producer's publish (SeqCst load, or take the mutex
/// the producer publishes under); a `true` aborts the park, (3) block.
pub(crate) fn park(
&self,
id: usize,
deadline: Option<Instant>,
recheck: impl FnOnce() -> bool,
) -> ParkResult {
debug_assert!(id < self.parkers.len(), "park: scheduler id out of range");
let bit = 1u64 << id;
// (1) publish. AcqRel RMW: the acquire half is the handshake — if
// this lands after a producer's mask RMW in modification order, we
// read-from it and the producer's earlier work publication is
// visible to the re-check below (see module docs).
let prev = self.idle.fetch_or(bit, Ordering::AcqRel);
debug_assert_eq!(prev & bit, 0, "park: idle bit already set for this id");
// Fence half of the producer handshake (see `wake_one_if_idle`):
// orders our bit-publish before the re-check's loads, so it pairs
// with the producer's publish→fence→mask-load — at least one side
// must see the other's store, whichever queue backend is in play.
fence(Ordering::SeqCst);
// (2) the mandatory post-publish re-check.
if recheck() {
self.idle.fetch_and(!bit, Ordering::AcqRel);
return ParkResult::WorkFound;
}
// (3) block. The permit closes the window between the re-check and
// the futex wait: a wake_one that picked us in that window has
// already CASed our bit clear and set the permit.
let woken = self.parkers[id].park(deadline);
// Clear own bit — a no-op when a waker already CASed it clear.
self.idle.fetch_and(!bit, Ordering::AcqRel);
if woken {
ParkResult::Woken
} else {
ParkResult::TimedOut
}
}
/// Wake exactly one parked scheduler, if any: pick the highest set idle
/// bit (LIFO-ish — warmest cache), CAS it clear, deliver a permit.
/// Empty mask = no-op (everyone is awake and will find work by
/// popping). Returns whether a scheduler was woken.
pub(crate) fn wake_one(&self) -> bool {
// RMW read, not a load: reads the latest mask by modification-order
// coherence, closing the Dekker race with a parking consumer (see
// module docs). The release side of the RMW is what a later-parking
// consumer's fetch_or acquires to make its re-check sound.
let mut mask = self.idle.fetch_or(0, Ordering::AcqRel);
loop {
if mask == 0 {
return false;
}
let id = 63 - mask.leading_zeros() as usize; // highest set bit
let bit = 1u64 << id;
// The CAS is the exactly-one guarantee: whoever clears the bit
// owns the wake; a racing wake_one retries on the observed value
// (coherence: a failed CAS can never read older than `mask`).
match self.idle.compare_exchange(
mask,
mask & !bit,
Ordering::AcqRel,
Ordering::Acquire,
) {
Ok(_) => {
self.parkers[id].unpark();
return true;
}
Err(m) => mask = m,
}
}
}
/// The producer-side wake tail (`enqueue`'s fast path, RFC 018 "enqueue
/// wakes"). The caller has just published work (queue push); we fence,
/// then read the mask with ONE Relaxed load — 0 means every scheduler
/// is awake and the pure-compute hot path pays no RMW on the shared
/// mask line. Soundness is the fence handshake (module docs): our
/// fence orders the caller's push before the mask load; the consumer's
/// fence (in `park`) orders its bit-publish before its re-check — at
/// least one side must observe the other's store, so a consumer we
/// miss here is a consumer whose re-check finds the caller's work.
pub(crate) fn wake_one_if_idle(&self) -> bool {
fence(Ordering::SeqCst);
if self.idle.load(Ordering::Relaxed) == 0 {
return false;
}
self.wake_one()
}
/// Terminal wake (replaces the AllDone wake-pipe byte): clear the mask
/// and deliver a permit to *every* parker, parked or not. A permit set
/// on a busy scheduler costs one spurious park return — nothing at the
/// terminal boundary. Idempotent.
pub(crate) fn wake_all(&self) {
self.idle.store(0, Ordering::Release);
for p in self.parkers.iter() {
p.unpark();
}
}
/// Latest idle mask (RMW read — same handshake as `wake_one`). A
/// test-only observer: production expresses the chain rule through
/// `wake_one_if_idle` (fence + Relaxed load), not a mask read.
#[cfg(test)]
pub(crate) fn idle_mask(&self) -> u64 {
self.idle.fetch_or(0, Ordering::AcqRel)
}
// ----- timekeeper -----
/// Try to take the timekeeper role for scheduler `id` with `deadline`.
/// MUST be called under the caller's timer serialization (the timers
/// mutex), with `deadline` the heap minimum peeked under that same
/// hold — this is what makes the insert-check race-free. Returns
/// whether the role was taken (false = someone else holds it; park
/// with no deadline).
pub(crate) fn try_arm_timer(&self, id: usize, deadline: Instant) -> bool {
debug_assert!(id < self.parkers.len(), "try_arm_timer: id out of range");
if self
.tk_holder
.compare_exchange(NO_TIMEKEEPER, id as u64, Ordering::SeqCst, Ordering::SeqCst)
.is_err()
{
return false;
}
// Holder-then-deadline order: an insert-check that sees the holder
// with the deadline still NO_DEADLINE compares `new < MAX` = true
// and over-wakes — the benign direction. (Under the mandated timer
// serialization this interleaving cannot occur anyway.)
self.tk_armed.store(self.deadline_nanos(deadline), Ordering::SeqCst);
true
}
/// Release the timekeeper role (the holder, on wake, before it
/// re-peeks/fires). Callable without the timer serialization: a
/// racing insert may wake a no-longer-holder — over-wake, benign.
pub(crate) fn disarm_timer(&self, id: usize) {
debug_assert_eq!(
self.tk_holder.load(Ordering::SeqCst),
id as u64,
"disarm_timer by a non-holder"
);
// Deadline first: a concurrent insert-check then sees NO_DEADLINE
// and skips the (now pointless) wake instead of waking a stale
// holder id. Either order is correct; this one wastes less.
self.tk_armed.store(NO_DEADLINE, Ordering::SeqCst);
self.tk_holder.store(NO_TIMEKEEPER, Ordering::SeqCst);
}
/// Insert-side re-arm check: if `deadline` is earlier than the armed
/// snapshot, wake the timekeeper to re-peek. MUST be called under the
/// same timer serialization as `try_arm_timer` (see there).
pub(crate) fn timer_inserted(&self, deadline: Instant) {
if self.deadline_nanos(deadline) < self.tk_armed.load(Ordering::SeqCst) {
let holder = self.tk_holder.load(Ordering::SeqCst);
if holder != NO_TIMEKEEPER {
// Direct unpark, not wake_one: the wake targets the
// timekeeper specifically (it must re-peek the heap). Its
// idle bit stays set until it returns from park — a
// concurrent wake_one may pick it too; over-wake, benign.
self.parkers[holder as usize].unpark();
}
}
}
/// The armed-deadline snapshot (nanos since origin; `NO_DEADLINE` =
/// none). Test-only introspection on the timekeeper's armed value; the
/// busy-path due-check reads `next_deadline`, not this.
#[cfg(test)]
pub(crate) fn armed_deadline_nanos(&self) -> u64 {
self.tk_armed.load(Ordering::Relaxed)
}
// ----- earliest-deadline snapshot (busy-path due-check) -----
/// Record a newly inserted timer deadline. MUST be called under the
/// timers mutex (same serialization rule as `try_arm_timer`), which is
/// why plain compare+store suffices for the min-maintenance. Also runs
/// the timekeeper re-arm check (`timer_inserted`) — one call site for
/// both consequences of an insert.
pub(crate) fn note_deadline(&self, deadline: Instant) {
let n = self.deadline_nanos(deadline);
if n < self.next_deadline.load(Ordering::Relaxed) {
self.next_deadline.store(n, Ordering::Release);
}
self.timer_inserted(deadline);
}
/// Re-anchor the snapshot to the heap minimum (`None` = heap empty)
/// after a `pop_due` / `clear`. MUST be called under the timers mutex.
pub(crate) fn refresh_deadline(&self, next: Option<Instant>) {
let n = next.map_or(NO_DEADLINE, |d| self.deadline_nanos(d));
self.next_deadline.store(n, Ordering::Release);
}
/// Busy-path due-check: is the earliest known deadline at or past
/// `now`? One Relaxed load when no deadline is armed — the clock is
/// read only when a timer actually exists (matching the old drain
/// phase's is_empty guard), so the pure-compute hot path pays a load
/// and a branch.
pub(crate) fn deadline_due(&self) -> bool {
let n = self.next_deadline.load(Ordering::Relaxed);
n != NO_DEADLINE && self.deadline_nanos(Instant::now()) >= n
}
/// The earliest-deadline snapshot as an `Instant` (`None` = no timer
/// pending). Test-only: the idle path arms the timekeeper from the
/// timer heap's own `peek_deadline` under the timers mutex.
#[cfg(test)]
pub(crate) fn next_deadline_instant(&self) -> Option<Instant> {
let n = self.next_deadline.load(Ordering::Acquire);
if n == NO_DEADLINE {
None
} else {
self.origin.checked_add(std::time::Duration::from_nanos(n))
}
}
}
// ---------------------------------------------------------------------------
// Unit tests (std build)
// ---------------------------------------------------------------------------
#[cfg(all(test, not(loom)))]
mod tests {
use super::*;
use std::sync::atomic::{AtomicUsize, Ordering as O};
use std::sync::Arc;
use std::time::{Duration, Instant};
#[test]
fn permit_before_park_returns_immediately() {
let c = Coordinator::new(1);
// Deliver the wake first (nobody parked: wake_one no-ops on the
// mask, so use the timekeeper-direct path? No — permit semantics
// are the parker's own; exercise via wake_all which permits all).
c.wake_all();
let t0 = Instant::now();
let r = c.park(0, None, || false);
assert_eq!(r, ParkResult::Woken);
assert!(t0.elapsed() < Duration::from_millis(100), "park blocked despite permit");
}
#[test]
fn submillisecond_deadline_is_honored() {
// Regression for the retired as_millis truncation: a 500µs deadline
// must neither busy-return instantly forever nor round to 0/∞.
let c = Coordinator::new(1);
let t0 = Instant::now();
let r = c.park(0, Some(t0 + Duration::from_micros(500)), || false);
let dt = t0.elapsed();
assert_eq!(r, ParkResult::TimedOut);
assert!(dt >= Duration::from_micros(400), "woke too early: {dt:?}");
assert!(dt < Duration::from_millis(50), "overslept: {dt:?}");
}
#[test]
fn recheck_true_aborts_park_and_clears_bit() {
let c = Coordinator::new(2);
let r = c.park(1, None, || true);
assert_eq!(r, ParkResult::WorkFound);
assert_eq!(c.idle_mask(), 0, "bit not cleared after WorkFound");
}
#[test]
fn recheck_observes_own_bit_published() {
let c = Coordinator::new(2);
let seen = std::cell::Cell::new(0u64);
let r = c.park(1, None, || {
seen.set(c.idle_mask());
true
});
assert_eq!(r, ParkResult::WorkFound);
assert_eq!(seen.get() & 0b10, 0b10, "bit not published before re-check");
}
#[test]
fn wake_one_wakes_exactly_one_of_n() {
const N: usize = 4;
let c = Arc::new(Coordinator::new(N));
let woken = Arc::new(AtomicUsize::new(0));
let mut ts = Vec::new();
for id in 0..N {
let c = c.clone();
let woken = woken.clone();
ts.push(std::thread::spawn(move || {
let r = c.park(id, None, || false);
assert_eq!(r, ParkResult::Woken);
woken.fetch_add(1, O::SeqCst);
}));
}
// Wait until all four are published idle.
let t0 = Instant::now();
while c.idle_mask().count_ones() != N as u32 {
assert!(t0.elapsed() < Duration::from_secs(5), "threads never parked");
std::thread::yield_now();
}
assert!(c.wake_one());
// Exactly one wakes; give the others a beat to (incorrectly) wake.
let t0 = Instant::now();
while woken.load(O::SeqCst) == 0 {
assert!(t0.elapsed() < Duration::from_secs(5), "wake_one woke nobody");
std::thread::yield_now();
}
std::thread::sleep(Duration::from_millis(100));
assert_eq!(woken.load(O::SeqCst), 1, "wake_one woke more than one");
assert_eq!(c.idle_mask().count_ones(), (N - 1) as u32);
c.wake_all();
for t in ts {
match t.join() {
Ok(()) => {}
Err(p) => std::panic::resume_unwind(p),
}
}
assert_eq!(woken.load(O::SeqCst), N);
assert_eq!(c.idle_mask(), 0);
}
#[test]
fn wake_one_prefers_highest_bit() {
let c = Arc::new(Coordinator::new(3));
let woken_id = Arc::new(AtomicUsize::new(usize::MAX));
let mut ts = Vec::new();
for id in 0..3 {
let c = c.clone();
let woken_id = woken_id.clone();
ts.push(std::thread::spawn(move || {
if c.park(id, None, || false) == ParkResult::Woken {
let _ = woken_id.compare_exchange(usize::MAX, id, O::SeqCst, O::SeqCst);
}
}));
}
let t0 = Instant::now();
while c.idle_mask() != 0b111 {
assert!(t0.elapsed() < Duration::from_secs(5));
std::thread::yield_now();
}
assert!(c.wake_one());
let t0 = Instant::now();
while woken_id.load(O::SeqCst) == usize::MAX {
assert!(t0.elapsed() < Duration::from_secs(5));
std::thread::yield_now();
}
assert_eq!(woken_id.load(O::SeqCst), 2, "LIFO-ish: highest bit first");
c.wake_all();
for t in ts {
let _ = t.join();
}
}
#[test]
fn wake_one_on_empty_mask_is_noop() {
let c = Coordinator::new(2);
assert!(!c.wake_one());
assert_eq!(c.idle_mask(), 0);
}
#[test]
fn timekeeper_arm_is_exclusive_and_snapshot_readable() {
let c = Coordinator::new(2);
let d2 = Instant::now() + Duration::from_secs(10);
let d1 = Instant::now() + Duration::from_secs(1);
assert_eq!(c.armed_deadline_nanos(), NO_DEADLINE);
assert!(c.try_arm_timer(0, d2));
assert!(!c.try_arm_timer(1, d1), "second arm must fail while held");
assert_eq!(c.armed_deadline_nanos(), c.deadline_nanos(d2));
c.disarm_timer(0);
assert_eq!(c.armed_deadline_nanos(), NO_DEADLINE);
assert!(c.try_arm_timer(1, d1), "role must be re-takeable after disarm");
c.disarm_timer(1);
}
#[test]
fn earlier_insert_wakes_timekeeper() {
let c = Coordinator::new(2);
let far = Instant::now() + Duration::from_secs(60);
let near = Instant::now() + Duration::from_millis(1);
assert!(c.try_arm_timer(0, far));
// Holder not yet parked: the wake must land as a permit.
c.timer_inserted(near);
let t0 = Instant::now();
let r = c.park(0, Some(far), || false);
assert_eq!(r, ParkResult::Woken, "re-arm wake lost");
assert!(t0.elapsed() < Duration::from_secs(5), "slept toward the stale deadline");
c.disarm_timer(0);
}
#[test]
fn later_insert_does_not_wake_timekeeper() {
let c = Coordinator::new(2);
let near = Instant::now() + Duration::from_millis(20);
let far = Instant::now() + Duration::from_secs(60);
assert!(c.try_arm_timer(0, near));
c.timer_inserted(far); // later than armed: no wake
let r = c.park(0, Some(near), || false);
assert_eq!(r, ParkResult::TimedOut, "spurious wake for a later insert");
c.disarm_timer(0);
}
#[test]
#[should_panic(expected = "1..=64")]
fn more_than_64_schedulers_asserts() {
let _ = Coordinator::new(65);
}
#[test]
fn wake_one_if_idle_noop_on_empty_and_wakes_on_parked() {
let c = Arc::new(Coordinator::new(1));
assert!(!c.wake_one_if_idle(), "empty mask must be a no-op");
let c2 = c.clone();
let t = std::thread::spawn(move || {
assert_eq!(c2.park(0, None, || false), ParkResult::Woken);
});
let t0 = Instant::now();
while c.idle_mask() == 0 {
assert!(t0.elapsed() < Duration::from_secs(5), "never parked");
std::thread::yield_now();
}
assert!(c.wake_one_if_idle());
match t.join() {
Ok(()) => {}
Err(p) => std::panic::resume_unwind(p),
}
}
#[test]
fn deadline_snapshot_min_maintenance_and_due_check() {
let c = Coordinator::new(1);
assert!(!c.deadline_due(), "no deadline: never due");
assert_eq!(c.next_deadline_instant(), None);
let far = Instant::now() + Duration::from_secs(60);
let near = Instant::now() + Duration::from_millis(1);
c.note_deadline(far);
assert!(!c.deadline_due());
c.note_deadline(near); // min wins
assert!(c.next_deadline_instant().is_some_and(|d| d <= near));
c.note_deadline(far); // later insert must NOT raise the snapshot
assert!(c.next_deadline_instant().is_some_and(|d| d <= near));
std::thread::sleep(Duration::from_millis(2));
assert!(c.deadline_due(), "past deadline not reported due");
c.refresh_deadline(Some(far));
assert!(!c.deadline_due(), "refresh did not re-anchor");
c.refresh_deadline(None);
assert_eq!(c.next_deadline_instant(), None);
// A deadline at/before origin encodes as 0: always due.
c.note_deadline(Instant::now() - Duration::from_secs(1));
assert!(c.deadline_due());
}
#[test]
fn note_deadline_earlier_wakes_timekeeper_via_snapshot_path() {
// note_deadline must carry the timer_inserted re-arm wake too.
let c = Coordinator::new(2);
let far = Instant::now() + Duration::from_secs(60);
let near = Instant::now() + Duration::from_millis(1);
assert!(c.try_arm_timer(0, far));
c.note_deadline(near);
let r = c.park(0, Some(far), || false);
assert_eq!(r, ParkResult::Woken, "re-arm wake lost through note_deadline");
c.disarm_timer(0);
}
}
// ---------------------------------------------------------------------------
// loom models — RUSTFLAGS="--cfg loom" cargo test --lib --release park
// ---------------------------------------------------------------------------
#[cfg(all(test, loom))]
mod loom_tests {
use super::*;
use loom::sync::atomic::{AtomicU64 as LAtomicU64, Ordering as O};
use loom::sync::{Arc, Mutex as LMutex};
use loom::thread;
use std::time::{Duration, Instant};
/// RFC 018 loom model 1 — no lost wake.
/// producer{publish item; wake_one} ∥ consumer{set bit; re-check; park}:
/// the consumer always observes the item or a permit; it can never
/// sleep past a published item (loom's deadlock detector is the
/// assertion — a consumer parked forever fails the model).
#[test]
fn no_lost_wake() {
loom::model(|| {
let c = Arc::new(Coordinator::new(1));
let item = Arc::new(LAtomicU64::new(0));
let prod = {
let c = c.clone();
let item = item.clone();
thread::spawn(move || {
// The enqueue shape: publish, then the fenced fast-path
// wake tail (this is what the runtime's enqueue calls).
item.store(1, O::SeqCst);
c.wake_one_if_idle();
})
};
// Consumer: loop until the item is popped. Guarded load+CAS,
// not a blind swap — a swap writes 0 even when empty, and
// coherence allows that write to land after the producer's
// store in modification order, destroying the item (a model
// bug loom caught in an earlier draft of this test).
loop {
if item.load(O::SeqCst) == 1
&& item.compare_exchange(1, 0, O::SeqCst, O::SeqCst).is_ok()
{
break;
}
let _ = c.park(0, None, || item.load(O::SeqCst) == 1);
}
prod.join().unwrap();
});
}
/// RFC 018 loom model 2 — chain propagation.
/// Two items, two sleepers, ONE producer wake: the chain rule (a woken
/// consumer that sees surplus work and a non-empty mask wakes again)
/// must get both items consumed with no further producer action.
#[test]
fn chain_propagation() {
loom::model(|| {
let c = Arc::new(Coordinator::new(2));
let items = Arc::new(LAtomicU64::new(0));
let consumed = Arc::new(LAtomicU64::new(0));
let mut hs = Vec::new();
for id in 0..2usize {
let c = c.clone();
let items = items.clone();
let consumed = consumed.clone();
hs.push(thread::spawn(move || loop {
if consumed.load(O::SeqCst) == 2 {
c.wake_all(); // release a sibling still parked
break;
}
let cur = items.load(O::SeqCst);
if cur > 0
&& items
.compare_exchange(cur, cur - 1, O::SeqCst, O::SeqCst)
.is_ok()
{
consumed.fetch_add(1, O::SeqCst);
// THE CHAIN RULE, exactly as production expresses it
// (runtime.rs schedule_loop): surplus ⇒ the fenced
// fast-path wake. Models the Relaxed-load chain path,
// not just the RMW one.
if items.load(O::SeqCst) > 0 {
c.wake_one_if_idle();
}
continue;
}
let _ = c.park(id, None, || {
items.load(O::SeqCst) > 0 || consumed.load(O::SeqCst) == 2
});
}));
}
// Producer (main): two items, ONE wake, via the enqueue-shaped
// fenced fast path.
items.store(2, O::SeqCst);
c.wake_one_if_idle();
for h in hs {
h.join().unwrap();
}
assert_eq!(consumed.load(O::SeqCst), 2);
assert_eq!(items.load(O::SeqCst), 0);
});
}
/// RFC 018 loom model 3 — timekeeper handoff.
/// An earlier-deadline insert racing the parking timekeeper: the
/// earlier deadline is always honored — either the timekeeper armed it
/// directly (insert landed first under the timers lock) or the insert
/// wakes the timekeeper to re-peek. A timekeeper sleeping toward the
/// stale later deadline would deadlock the model (loom has no time).
#[test]
fn timekeeper_handoff() {
loom::model(|| {
let origin = Instant::now();
let d_far = origin + Duration::from_secs(60);
let d_near = origin + Duration::from_secs(1);
let c = Arc::new(Coordinator::new(1));
// The timers-mutex stand-in: heap min under a lock.
let heap_min = Arc::new(LMutex::new(d_far));
let tk = {
let c = c.clone();
let heap_min = heap_min.clone();
thread::spawn(move || {
// Peek + arm under the lock (the serialization rule).
let armed = {
let g = heap_min.lock().unwrap();
let min = *g;
assert!(c.try_arm_timer(0, min));
min
};
if armed == d_far {
// Insert hadn't landed: it MUST wake us. Parking
// toward d_far with no wake = model deadlock.
let r = c.park(0, Some(armed), || false);
assert_eq!(r, ParkResult::Woken, "re-arm wake lost");
}
// Woken (or armed the near deadline directly): re-peek.
c.disarm_timer(0);
let g = heap_min.lock().unwrap();
assert_eq!(*g, d_near, "earlier deadline not visible on re-peek");
})
};
// Inserter (main): publish the earlier deadline under the lock,
// then the insert-check.
{
let mut g = heap_min.lock().unwrap();
*g = d_near;
c.timer_inserted(d_near);
}
tk.join().unwrap();
});
}
/// RFC 018 loom model 4 — termination.
/// The AllDone verdict: producer flips done and wake_all()s; consumers
/// must never park forever past done (park's re-check + wake_all's
/// permits close every interleaving; a stuck consumer = loom deadlock).
#[test]
fn termination_no_park_past_done() {
loom::model(|| {
let c = Arc::new(Coordinator::new(2));
let done = Arc::new(LAtomicU64::new(0));
let mut hs = Vec::new();
for id in 0..2usize {
let c = c.clone();
let done = done.clone();
hs.push(thread::spawn(move || loop {
if done.load(O::SeqCst) == 1 {
break;
}
let _ = c.park(id, None, || done.load(O::SeqCst) == 1);
}));
}
done.store(1, O::SeqCst);
c.wake_all();
for h in hs {
h.join().unwrap();
}
});
}
}
+7 -4
View File
@@ -98,10 +98,13 @@ pub(crate) fn clear_current_slot() {
CURRENT_SLOT.with(|c| c.set(std::ptr::null()));
}
/// RFC 007 (`smarm-causal`) — raw pointer to the on-CPU actor's slot, null on
/// the scheduler's own stack. Same lifetime argument as `note_overrun`: the
/// slot is never reclaimed while its actor is on-CPU.
#[cfg(feature = "smarm-causal")]
/// Raw pointer to the on-CPU actor's slot, null on the scheduler's own
/// stack. Same lifetime argument as `note_overrun`: the slot is never
/// reclaimed while its actor is on-CPU. Consumers: the `smarm-causal`
/// profiler (RFC 007) and — unconditionally — the SIGSEGV classifier
/// (RFC 019 §7), which additionally relies on this being a plain load of a
/// const-initialized TLS Cell (no lazy init, no allocation, no dtor): safe
/// from a signal handler.
#[inline]
pub(crate) fn current_slot_ptr() -> *const crate::runtime::Slot {
CURRENT_SLOT.with(|c| c.get())
+242 -167
View File
@@ -1,55 +1,108 @@
//! Named mailbox registry — resolve a name (or pid) to a *messageable* actor.
//! Give an actor a name so other actors can find it and message it.
//!
//! ## What changed (RFC 014)
//! Without the registry, the only way to reach an actor is to already be
//! holding its [`Pid`], usually because you spawned it yourself or someone
//! passed it to you. That is fine for a worker you just created, but it does
//! not work for a well-known service that arbitrary parts of your program
//! need to find independently, like a logger, a config store, or a
//! connection pool. The registry solves this: an actor claims a name once,
//! and from then on any other actor can look that name up, or send to it
//! directly, without ever having been handed a `Pid`.
//!
//! The old registry was a `name <-> pid` bimap: `whereis` handed back a `Pid`
//! you could not send to, because a pid is just `(index, generation)` with no
//! delivery endpoint. This rework makes resolution yield something messageable.
//! ```
//! use smarm::{channel, register, run, send, spawn, unregister, whereis, Name};
//!
//! Two facts shape the structure:
//! const COUNTER: Name<u64> = Name::new("counter");
//!
//! 1. **A name resolves to a single actor.** Many actors under one label is
//! what *process groups* (`pg`) are for; the registry is one-name-one-actor
//! (several names *may* point at the same actor).
//! 2. **Channels are typed**, so an actor has no single untyped mailbox. An
//! actor instead owns a *set* of typed channels — one [`Sender`] per message
//! type it accepts. So the registry maps name/pid to a [`Mailbox`]: a small
//! structure holding that actor's pid plus all of its typed channels, keyed
//! by message [`TypeId`].
//! run(|| {
//! let (ready_tx, ready_rx) = channel::<()>();
//! let (tx, rx) = channel::<u64>();
//!
//! Resolution is therefore: `name -> pid` (single actor) `-> Mailbox -> the
//! channel for message type M`. A `Name<Cmd>` and a `Name<Admin>` on the *same*
//! actor select *different* channels purely by their type parameter, so
//! capability separation (RFC 014 §4.7) needs no extra machinery.
//! let worker = spawn(move || {
//! // Claim the name for this actor's inbox. Any actor holding
//! // `COUNTER` can now reach this one by name.
//! register(COUNTER, tx).unwrap();
//! ready_tx.send(()).unwrap();
//! assert_eq!(rx.recv().unwrap(), 42);
//! });
//!
//! ## Type erasure is contained
//! ready_rx.recv().unwrap(); // wait for the worker to register
//!
//! Each stored channel is a `Box<dyn Any + Send>` that is concretely a
//! `Sender<M>`, filed under `TypeId::of::<M>()`. A resolve for `M` looks up
//! that exact `TypeId` and downcasts to `Sender<M>` — keyed by the very type we
//! downcast to, so the downcast cannot fail on correct data; a failure is a
//! smarm bug, asserted in debug. The phantom `M` on [`Name`] re-imposes the
//! type at the call site, so callers never touch the erasure.
//! // Look the name up, or just send to it directly.
//! assert_eq!(whereis("counter"), Some(worker.pid()));
//! send(COUNTER, 42).unwrap();
//!
//! ## Cleanup is lazy (prune-on-contact)
//! worker.join().unwrap();
//!
//! As before, there is no `finalize` hook and no name field on the slot. Every
//! operation that touches a binding checks the target pid's liveness via the
//! generation-checked slot word; a binding to a dead actor behaves as absent
//! and is pruned on contact (its [`Mailbox`] and every name pointing at it are
//! dropped). The cost is a dead binding lingering until something looks at it;
//! the payoff is zero coupling to the actor lifecycle.
//! // The name dies with the actor: nobody holds it anymore.
//! assert_eq!(whereis("counter"), None);
//! });
//! ```
//!
//! ## Locking
//! ## Names carry a message type
//!
//! One `RawMutex` (Leaf class) in `RuntimeInner`, exactly like the old
//! registry. The fold (name index *and* handles under the one lock) is what
//! keeps a name-addressed `send` on a single Leaf — `raw_mutex` panics on a
//! second Leaf acquired while one is held. The send path clones the `Sender`
//! **under** the Leaf lock (a `Sender::clone` takes a Channel lock, permitted
//! under a Leaf), then **releases** the Leaf and only *then* sends — a send can
//! unpark a receiver, and wakeup-bearing work runs outside the Leaf. Order is
//! **Leaf -> Channel**, as `pg`/`finalize`.
//! A [`Name<M>`] is a plain string plus a type parameter `M`: the message
//! type that name expects to receive. [`Name::new`] is `const`, so the usual
//! pattern is a module-level constant like `COUNTER` above, shared by every
//! caller. The type parameter means a name is only ever sent the kind of
//! message it was declared for. If two different constants share the same
//! string but have different message types, they still address two
//! independent channels on the same actor: registering both just gives that
//! actor two ways to be reached, one per message type. This is how you give
//! one actor a "public" channel and a separate, differently-typed "admin"
//! channel under related names, without inventing an enum to merge them.
//!
//! ## One actor per name, looked up fresh every time
//!
//! A name always points at exactly one actor at a time (contrast a *process
//! group*, from the [`pg`](crate::pg) module, which is one name mapping to
//! many actors). Unlike a plain [`Pid`], which names one specific actor
//! forever and stops working the moment that actor dies, a name is
//! re-resolved on every [`send`]: if the actor holding it dies and a new one
//! registers under the same name, the next `send` reaches the new holder
//! automatically. Use a name for a long-lived service whose exact identity
//! you do not want to track by hand; use a `Pid` when you already have one
//! and want to talk to that exact actor.
//!
//! ## Registration ends when the actor does
//!
//! There is no separate step to clean up a name when its actor exits: dying
//! is enough. The next operation that touches a dead binding (a [`whereis`],
//! a [`send`], or another actor's [`register`] of the same name) notices the
//! actor is gone and clears the stale entry as a side effect, so the name
//! becomes free again. [`unregister`] is only for a live actor voluntarily
//! giving up a name it no longer wants; nothing has to call it on the way
//! out.
//!
//! ## Implementation notes
//!
//! These details matter if you are working on smarm itself; they are not
//! part of the public contract.
//!
//! Internally, each live actor that has published at least one channel owns
//! a `Mailbox`: its pid plus a set of typed channels, keyed by the message
//! type's `TypeId`. A stored channel is a `Box<dyn Any + Send>` that
//! is concretely a `Sender<M>`; resolving for `M` looks up that exact
//! `TypeId` and downcasts, so the downcast cannot fail on correct data (a
//! failure would be a bug in the registry itself, checked in debug builds).
//! Registering a name therefore means: find or create the actor's mailbox,
//! insert the channel under its type, and point the name at the actor's pid.
//!
//! There is no callback when an actor exits. Every operation that touches a
//! binding checks the target pid's liveness directly against the scheduler's
//! slot table (which also tracks a generation counter, so a dead actor's
//! reused slot index is never mistaken for the same actor). A binding to a
//! dead actor is treated as absent and dropped right there. This keeps the
//! registry decoupled from actor teardown, at the cost of a dead binding
//! lingering until something happens to look at it.
//!
//! The whole registry (both the name index and the per-actor mailboxes) sits
//! behind one lock, which is what lets a name-addressed [`send`] resolve and
//! clone the target's sender in a single critical section. The sender is
//! cloned while that lock is held, then the lock is released before the
//! actual send, since delivering a message can wake a parked receiver and
//! that wakeup work should not run while the registry is locked.
use crate::channel::Sender;
use crate::pid::{Addressable, Name, Pid};
@@ -80,28 +133,33 @@ impl std::fmt::Display for RegisterError {
impl std::error::Error for RegisterError {}
/// Why a name-addressed [`send`] did not deliver. Carries the message back so
/// the caller never loses it (mirrors [`crate::channel::SendError`]).
/// Why a send did not deliver. Every variant carries the undelivered message
/// back, mirroring [`crate::channel::SendError`], so a failed send never
/// silently drops what you tried to send.
///
/// `Debug`/`Display` are hand-written so neither demands `M: Debug` — the
/// payload is returned, not printed.
/// `Debug` and `Display` are hand-written so neither requires `M: Debug`,
/// since the payload is handed back to you, not printed.
pub enum SendError<M> {
/// No live actor is currently registered under this name. Name-addressed
/// [`send`] only; the pid-addressed counterpart is [`SendError::Dead`].
/// No live actor is currently registered under this name. Returned only
/// by name-addressed [`send`]; the pid-addressed counterpart of "nothing
/// there" is [`SendError::Dead`].
Unresolved(M),
/// The pid-addressed actor is no longer the live incarnation this pid names
/// — it has died, even if its slot now holds a *different* actor (a direct
/// `Pid<A>` send never redirects; contrast name-addressed [`send`]). Pid
/// paths ([`send_to`] / [`send_dyn`]) only.
/// The actor this pid identifies has died, even if its slot has since
/// been taken over by a different, live actor. A direct `Pid<A>` send
/// never redirects to that new occupant; contrast name-addressed
/// [`send`], which would reach it. Returned by the pid-addressed sends,
/// [`send_to`] and [`send_dyn`].
Dead(M),
/// The actor is live but exposes no channel for this message type.
/// The actor is live but has not published a channel for this message
/// type.
NoChannel(M),
/// The actor's channel for this message type is closed (its receiver is gone).
/// The actor's channel for this message type is closed (its receiver has
/// been dropped).
Closed(M),
/// No live member to deliver to — a [`dispatch`](crate::dispatch) over an
/// empty (or all-dead) process group. Group-addressed dispatch only; the
/// name-addressed counterpart is [`SendError::Unresolved`]. The message is
/// handed back undelivered.
/// No live member was available to deliver to: returned by
/// [`dispatch`](crate::dispatch) when the target process group is empty
/// or every member in it has died. The name-addressed counterpart of
/// this case is [`SendError::Unresolved`].
NoMember(M),
}
@@ -150,9 +208,9 @@ impl<M> std::error::Error for SendError<M> {}
/// A registry-stored channel, type-erased over its message type. The stored
/// object must serve two readers: `clone_sender` (downcast back to the concrete
/// `Sender<M>`) and the RFC 016 snapshot (queued length without knowing `M`).
/// A bare `Box<dyn Any>` gives the first but not the second, so we erase behind
/// this small trait instead.
/// `Sender<M>`) and the runtime introspection snapshot (queued length without
/// knowing `M`). A bare `Box<dyn Any>` gives the first but not the second, so
/// we erase behind this small trait instead.
trait ErasedSender: Send {
fn as_any(&self) -> &dyn Any;
fn queued_len(&self) -> usize;
@@ -169,7 +227,7 @@ impl<M: Send + 'static> ErasedSender for Sender<M> {
/// One typed channel of an actor, type-erased. Concretely a `Sender<M>` filed
/// under `TypeId::of::<M>()`; `msg_type` is `type_name::<M>()`, kept for
/// observers (RFC 014 §4.5) and as the debug cross-check on the downcast.
/// observability tooling and as the debug cross-check on the downcast.
struct Channel {
sender: Box<dyn ErasedSender>,
msg_type: &'static str,
@@ -204,7 +262,7 @@ impl Mailbox {
}
}
/// Per-actor registry view handed to RFC 016 introspection: registered names
/// Per-actor registry view handed to runtime introspection: registered names
/// and summed mailbox depth, tagged with the mailbox's `pid` so a stale
/// incarnation can be filtered against the slab. Covers only *published*
/// channels (`register` / `install` / `spawn_addr` / gen_server start); an
@@ -219,17 +277,18 @@ pub(crate) struct MailboxInfo {
/// The directory. Invariant (held under the registry lock): every value in
/// `by_name` is the full [`Pid`] (index *and* generation) of an actor that
/// published a [`Mailbox`] into `by_index` at registration time. Stale entries
/// (dead holders — including holders whose slot has since been re-tenanted by
/// a different actor) violate nothing — they are pruned on contact, and the
/// (dead holders, including holders whose slot has since been re-tenanted by
/// a different actor) violate nothing: they are pruned on contact, and the
/// generation makes "dead" decidable even after slot reuse.
pub(crate) struct Registry {
/// `pid.index() -> the actor's mailbox`. The handle store.
by_index: HashMap<u32, Mailbox>,
/// `name -> holder pid`. Several names may map to one actor. The full pid
/// (not just the index) is load-bearing: an index alone cannot tell a dead
/// holder from the live actor now tenanting its recycled slot, which made
/// such a name read as live-held — unresolvable *and* unregisterable — and
/// would misdeliver to a same-typed tenant (soak20 signature 2).
/// holder from the live actor now tenanting its recycled slot. Comparing
/// only the index would make such a name read as live-held (unresolvable
/// and unregisterable at once) and could misdeliver to whatever new,
/// same-typed actor now sits in that slot.
by_name: HashMap<&'static str, Pid>,
}
@@ -238,10 +297,10 @@ impl Registry {
Self { by_index: HashMap::new(), by_name: HashMap::new() }
}
/// Drop a dead holder's artifacts: every name bound to it, and its mailbox
/// — but only while the mailbox is still *its own*. A recycled slot's
/// mailbox belongs to the live tenant (publish replaces it wholesale on
/// pid mismatch) and is left untouched.
/// Drop a dead holder's artifacts: every name bound to it, and its
/// mailbox, but only while the mailbox is still *its own*. A recycled
/// slot's mailbox belongs to the live tenant (publish replaces it
/// wholesale on pid mismatch) and is left untouched.
fn prune_holder(&mut self, holder: Pid) {
self.by_name.retain(|_, p| *p != holder);
if self.by_index.get(&holder.index()).is_some_and(|mb| mb.pid == holder) {
@@ -249,17 +308,16 @@ impl Registry {
}
}
/// RFC 016 snapshot input: per-slot-index registry view — the actor's
/// registered names (inverted from `by_name`) and its mailbox depth (queued
/// messages summed across every published typed channel). Built in one pass
/// under the registry Leaf; the per-channel `queued_len` takes a Channel
/// lock, legal under the Leaf (Leaf → Channel). Carries each mailbox's full
/// `pid` so the caller can discard a stale incarnation's entry against the
/// slab's live generation. Names are matched to mailboxes by *full pid*, so
/// a stale name (dead holder) still annotates the corpse's own mailbox if
/// that survives, but never a recycled slot's new tenant; names that attach
/// to no mailbox are dropped — they violate no invariant and get pruned on
/// next contact.
/// Runtime introspection input: per-slot-index registry view, giving the
/// actor's registered names (inverted from `by_name`) and its mailbox
/// depth (queued messages summed across every published typed channel).
/// Carries each mailbox's full `pid` so the caller can discard a stale
/// incarnation's entry against the slab's live generation. Names are
/// matched to mailboxes by *full pid*, so a stale name (dead holder)
/// still annotates the corpse's own mailbox if that survives, but never a
/// recycled slot's new tenant; names that attach to no mailbox are
/// dropped, since that violates no invariant and they get pruned on next
/// contact.
pub(crate) fn introspect_map(&self) -> HashMap<u32, MailboxInfo> {
let mut names: HashMap<Pid, Vec<&'static str>> = HashMap::new();
for (&name, &pid) in &self.by_name {
@@ -282,8 +340,9 @@ impl Registry {
/// Single-actor form of [`introspect_map`](Self::introspect_map): the
/// registry view for one slot index, or `None` if no mailbox is published
/// there. Used by `actor_info` so its cost stays proportional to the one
/// actor rather than locking every channel in the runtime.
/// there. Used by the runtime's per-actor introspection so its cost stays
/// proportional to the one actor rather than locking every channel in the
/// runtime.
pub(crate) fn introspect_one(&self, idx: u32) -> Option<MailboxInfo> {
let mb = self.by_index.get(&idx)?;
let depth: usize = mb.channels.values().map(|c| c.sender.queued_len()).sum();
@@ -301,14 +360,18 @@ fn live(inner: &crate::runtime::RuntimeInner, pid: Pid) -> bool {
inner.slot_at(pid).is_some_and(|s| s.is_live_for(pid))
}
/// Publish the current actor's `Sender<M>` under `name`, capturing the channel
/// so the name becomes messageable. Idempotent for the same `(name, type)`;
/// registering a *second* type under the same (or another) name on the same
/// actor just adds another channel to the actor's mailbox.
/// Give the current actor's channel a name, so other actors can find and
/// message it by that name instead of needing its [`Pid`].
///
/// Fails with [`RegisterError::NameTaken`] if the name is held by a *different*
/// live actor (a binding to a dead actor is pruned and the name treated as
/// free). Panics if called outside `Runtime::run()`.
/// Calling this again with the same `(name, type)` from the same actor is
/// harmless. Registering a *second* message type under the same (or a
/// different) name from the same actor just adds another typed channel to
/// that actor's mailbox; it does not replace the first.
///
/// Fails with [`RegisterError::NameTaken`] if the name is currently held by a
/// *different* live actor. A name held by an actor that has since died is not
/// considered taken: it is quietly reclaimed and handed to you. Panics if
/// called outside [`run`](crate::run).
pub fn register<M: Send + 'static>(name: Name<M>, tx: Sender<M>) -> Result<(), RegisterError> {
register_with(self_pid(), name.as_str(), tx)
}
@@ -316,8 +379,8 @@ pub fn register<M: Send + 'static>(name: Name<M>, tx: Sender<M>) -> Result<(), R
/// Bind `name` to `pid`'s mailbox and publish `tx` under `M`'s [`TypeId`], for
/// an explicit (already-live) actor rather than `self`. The shared core of
/// [`register`] (which passes `self_pid()`) and the parent-side server-name
/// bind in `gen_server` (which names a freshly spawned server before its body
/// has run, so the name resolves the instant `start()` returns). Same collision
/// bind in `gen_server`, which names a freshly spawned server before its body
/// has run, so the name resolves the instant `start()` returns. Same collision
/// rules and lock discipline as `register`.
pub(crate) fn register_with<M: Send + 'static>(
me: Pid,
@@ -337,8 +400,8 @@ pub(crate) fn register_with<M: Send + 'static>(
} else {
// Dead holder: free the name (and its other stale artifacts).
// Liveness is judged against the *stored* pid, generation
// included — a recycled slot's live tenant no longer makes a
// dead name read as taken (soak20 signature 2).
// included, so a recycled slot's live tenant no longer makes a
// dead name read as taken.
reg.prune_holder(holder);
}
}
@@ -367,14 +430,14 @@ fn publish_channel<M: Send + 'static>(reg: &mut Registry, me: Pid, tx: Sender<M>
/// Publish the current actor's `Sender<A::Msg>` into its mailbox **without**
/// binding a name, and hand back the typed [`Pid<A>`] that addresses this
/// actor directly. This is the opt-in, lazy install of RFC 014 §5: an actor
/// that wants to be reachable by a direct, identity-bound [`Pid<A>`] (rather
/// than only via a re-resolving [`Name`]) calls this once with its inbox
/// sender, then hands the returned pid out.
/// actor directly.
///
/// Unlike [`register`] there is no name to collide on, and `self` is always a
/// live actor inside `run()`, so this is infallible. Panics if called outside
/// `Runtime::run()`.
/// This is for an actor that wants to be reachable directly by its pid,
/// rather than only through a re-resolving [`Name`]: call this once with your
/// inbox sender, then hand the returned `Pid<A>` to whoever should be able to
/// message you. Unlike [`register`] there is no name to collide on, and the
/// current actor is always live while inside `run()`, so this cannot fail.
/// Panics if called outside [`run`](crate::run).
pub fn install<A: Addressable>(tx: Sender<A::Msg>) -> Pid<A> {
let me = self_pid();
with_runtime(|inner| {
@@ -390,11 +453,12 @@ pub fn install<A: Addressable>(tx: Sender<A::Msg>) -> Pid<A> {
/// Publish `tx` into `pid`'s mailbox under `M`'s [`TypeId`], for an explicit
/// (freshly minted, already-live) actor rather than `self`. The parent-side
/// half of [`spawn_addr`](crate::spawn_addr): the spawner makes the inbox and
/// publishes the sender here *before* handing back the `Pid<A>`, so an immediate
/// `send_to` on the returned pid always resolves — the address is live the
/// instant the caller holds it, with no dependence on the body having run yet.
/// publishes the sender here *before* handing back the `Pid<A>`, so an
/// immediate `send_to` on the returned pid always resolves. The address is
/// live the instant the caller holds it, with no dependence on the spawned
/// actor's body having run yet.
///
/// Caller guarantees `pid` is the just-installed actor (Queued, this exact
/// Caller guarantees `pid` is the just-installed actor (queued, this exact
/// incarnation); `publish_channel` replaces any stale leftover at the slot.
pub(crate) fn install_for<M: Send + 'static>(pid: Pid, tx: Sender<M>) {
with_runtime(|inner| {
@@ -404,8 +468,9 @@ pub(crate) fn install_for<M: Send + 'static>(pid: Pid, tx: Sender<M>) {
});
}
/// The single actor currently registered under `name`, or `None` if unbound or
/// no longer live (the stale binding is pruned on the way out).
/// Look up which actor currently holds `name`, if any. Returns `None` if the
/// name is unbound, or if it was bound to an actor that has since died (the
/// stale binding is cleared as a side effect of this call).
pub fn whereis(name: &str) -> Option<Pid> {
with_runtime(|inner| {
let mut reg = inner.registry.lock();
@@ -421,34 +486,37 @@ pub fn whereis(name: &str) -> Option<Pid> {
})
}
/// Resolve `name` to a *typed* [`Pid<A>`] — the identity-bound counterpart of
/// [`whereis`] (RFC 014 §4.4). Recovers the compile-checked
/// [`send_to`] path from a durable name: looks the name up,
/// then re-types the erased pid as `Pid<A>` via the unchecked
/// [`assert_type`](crate::pid::assert_type) primitive. A wrong `A` is not
/// unsound — it degrades to [`SendError::NoChannel`] on the next send (routing
/// is by message `TypeId`), never a misdelivery. `None` if unbound or dead.
/// Like [`whereis`], but returns a *typed* [`Pid<A>`] instead of a bare
/// [`Pid`], so a follow-up [`send_to`] is compile-checked instead of needing
/// the untyped [`send_dyn`] escape hatch. `None` if the name is unbound or its
/// holder has died.
///
/// Panics if called outside `Runtime::run()`.
/// The type `A` is not checked against what the name's holder actually
/// published: if you pick the wrong `A`, this still succeeds, but the next
/// send against the returned pid degrades to [`SendError::NoChannel`] rather
/// than reaching the wrong actor or the wrong channel.
///
/// Panics if called outside [`run`](crate::run).
pub fn lookup_as<A: Addressable>(name: &str) -> Option<Pid<A>> {
whereis(name).map(crate::pid::assert_type::<A>)
}
/// Resolve `name` to its actor's pid and a cloned `Sender<M>`, under the Leaf
/// lock (clone-under-lock, then release). The crate-internal building block for
/// `gen_server`'s by-name addressing: a named server publishes its inbox as a
/// `Sender<Envelope<G>>` (via [`register_with`]), and `whereis_server` / `call`
/// / `cast` recover that exact typed sender here to rebuild a `GenServerRef<G>`.
/// `None` if unbound, dead (pruned on the way out), or holding no `M` channel.
/// Resolve `name` to its actor's pid and a cloned `Sender<M>`, all under one
/// lock acquisition. The crate-internal building block for `gen_server`'s
/// by-name addressing: a named server publishes its inbox as a
/// `Sender<Envelope<G>>` (via [`register_with`]), and the server's `call` /
/// `cast` / `whereis_server` recover that exact typed sender here to rebuild a
/// `GenServerRef<G>`. `None` if unbound, dead (pruned on the way out), or
/// holding no `M` channel.
pub(crate) fn resolve_named_sender<M: Send + 'static>(name: &str) -> Option<(Pid, Sender<M>)> {
with_runtime(|inner| {
let mut reg = inner.registry.lock();
let pid = *reg.by_name.get(name)?;
if !live(inner, pid) {
// Stored-pid liveness, generation included: a name whose holder
// died is pruned (heals) even if the slot has a new tenant —
// previously the tenant's mailbox made the name unresolvable
// *without* pruning, wedging it for the tenant's lifetime.
// died is pruned (heals) even if the slot has a new tenant.
// Otherwise the tenant's mailbox would make the name unresolvable
// without pruning, wedging it for the tenant's lifetime.
reg.prune_holder(pid);
return None;
}
@@ -459,9 +527,10 @@ pub(crate) fn resolve_named_sender<M: Send + 'static>(name: &str) -> Option<(Pid
})
}
/// Remove the binding for `name`, returning the actor it pointed at if still
/// live. Only the *name* is freed; the actor's mailbox (and any other names for
/// it) remain. A binding to a dead actor is reported as `None`.
/// Give up a name. Returns the actor it pointed at, if that actor was still
/// live. Only the *name* is freed; the actor's mailbox (and any other names
/// bound to it) are unaffected. A binding to an already-dead actor reports
/// `None`, since there was nothing live to release.
pub fn unregister(name: &str) -> Option<Pid> {
with_runtime(|inner| {
let mut reg = inner.registry.lock();
@@ -470,18 +539,21 @@ pub fn unregister(name: &str) -> Option<Pid> {
})
}
/// Resolve `name` to its actor's `Sender<M>` and deliver `msg`. The whole point
/// of the rework: a name you can *send* to.
/// Look `name` up and deliver `msg` to whichever actor currently holds it.
/// This is the point of naming an actor: a name you can send a message to
/// directly, without a separate lookup step.
///
/// Errors (message returned in every case): [`SendError::Unresolved`] if no
/// live actor holds the name, [`SendError::NoChannel`] if that actor has no
/// channel for `M`, [`SendError::Closed`] if its `M` channel's receiver is
/// gone. Panics if called outside `Runtime::run()`.
/// On failure the message comes back to you, wrapped in the [`SendError`]
/// variant that explains why: [`SendError::Unresolved`] if no live actor
/// currently holds the name, [`SendError::NoChannel`] if the actor that holds
/// it never published a channel for `M`, or [`SendError::Closed`] if it
/// published one but has since dropped the receiving end. Panics if called
/// outside [`run`](crate::run).
pub fn send<M: Send + 'static>(name: Name<M>, msg: M) -> Result<(), SendError<M>> {
let key = name.as_str();
with_runtime(|inner| {
// Resolve + clone the sender under the Leaf lock, then drop the lock
// before sending (a send can unpark a receiver).
// Resolve + clone the sender under the registry lock, then drop the
// lock before sending (a send can unpark a receiver).
let tx = {
let mut reg = inner.registry.lock();
let pid = match reg.by_name.get(key) {
@@ -489,11 +561,9 @@ pub fn send<M: Send + 'static>(name: Name<M>, msg: M) -> Result<(), SendError<M>
None => return Err(SendError::Unresolved(msg)),
};
if !live(inner, pid) {
// Stored-pid liveness (generation included). Previously this
// checked the slot's *current* mailbox pid, so a recycled
// slot's live tenant passed — and a same-typed tenant would
// have received the message (misdelivery), a differently
// typed one a misleading NoChannel.
// Stored-pid liveness (generation included), so a recycled
// slot's new live tenant is never mistaken for the name's
// original (now-dead) holder.
reg.prune_holder(pid);
return Err(SendError::Unresolved(msg));
}
@@ -508,18 +578,19 @@ pub fn send<M: Send + 'static>(name: Name<M>, msg: M) -> Result<(), SendError<M>
/// Resolve a *raw* pid to its mailbox and deliver `msg` on the channel for `M`,
/// with **no redirect**. The stored mailbox must be this exact incarnation
/// (generation included) and still live; otherwise the actor this pid named is
/// gone and the result is [`SendError::Dead`] — even when the slot now holds a
/// different, live actor (which we leave untouched). Shared by [`send_to`]
/// (typed, `M = A::Msg`, channel guaranteed on an installed actor) and
/// [`send_dyn`] (explicit `M`, where `NoChannel` is a real outcome).
/// (generation included) and still live; otherwise the actor this pid named
/// is gone and the result is [`SendError::Dead`], even when the slot now
/// holds a different, live actor (which is left untouched). Shared by
/// [`send_to`] (typed, `M = A::Msg`, channel guaranteed on an installed
/// actor) and [`send_dyn`] (explicit `M`, where `NoChannel` is a real
/// outcome).
fn send_to_pid<M: Send + 'static>(
inner: &crate::runtime::RuntimeInner,
pid: Pid,
msg: M,
) -> Result<(), SendError<M>> {
// Resolve + clone the sender under the Leaf lock, then drop the lock before
// sending (a send can unpark a receiver) — Leaf -> Channel, as name `send`.
// Resolve + clone the sender under the registry lock, then drop the lock
// before sending (a send can unpark a receiver), same order as `send`.
let tx = {
let mut reg = inner.registry.lock();
match reg.by_index.get(&pid.index()).map(|m| m.pid) {
@@ -543,32 +614,36 @@ fn send_to_pid<M: Send + 'static>(
tx.send(msg).map_err(|crate::channel::SendError(m)| SendError::Closed(m))
}
/// Deliver `msg` to the exact actor named by `pid` — RFC 014 §4.2's direct,
/// identity-bound addressing mode. Unlike name-addressed [`send`] there is **no
/// redirect**: if that incarnation has died the message comes back as
/// [`SendError::Dead`], even if its slot now holds a different actor.
/// Deliver `msg` directly to the exact actor identified by `pid`. Unlike
/// name-addressed [`send`], there is **no redirect**: if that specific actor
/// has died, the message comes back as [`SendError::Dead`], even if its slot
/// has since been taken over by a different, live actor. Use this when you
/// already hold a `Pid<A>` and want to talk to that one actor specifically;
/// use [`send`] with a [`Name`] when you want whichever actor currently holds
/// a name.
///
/// The message type is the actor's `A::Msg`, so on a live actor that has
/// installed its inbox (via [`install`] or [`register`]) the channel is always
/// present; [`SendError::NoChannel`] therefore means the actor is live but
/// never published a `Pid<A>`-reachable inbox. Panics if called outside
/// `Runtime::run()`.
/// installed its inbox (via [`install`] or [`register`]) the channel is
/// always present; [`SendError::NoChannel`] therefore means the actor is live
/// but never published a `Pid<A>`-reachable inbox. Panics if called outside
/// [`run`](crate::run).
pub fn send_to<A: Addressable>(pid: Pid<A>, msg: A::Msg) -> Result<(), SendError<A::Msg>> {
with_runtime(|inner| send_to_pid::<A::Msg>(inner, pid.erase(), msg))
}
/// The explicit bare-pid escape hatch (RFC 014 §4.6): deliver `msg` of type `M`
/// to `pid` when all you hold is an untyped [`Pid`] — a pid off a [`Down`], or
/// out of a future `members()` — so the typed [`send_to`] is unavailable.
/// The escape hatch for sending to a bare, untyped [`Pid`] when the typed
/// [`send_to`] is unavailable, for example a pid recovered from a [`Down`]
/// notification or a group's `members()` list, where you no longer know the
/// actor's message type at compile time.
///
/// This is the one send whose message type can genuinely be wrong: the actor
/// may be live yet expose no channel for `M`, returning [`SendError::NoChannel`]
/// (on the typed paths that downcast collapses to a `debug_assert`). It is
/// named and documented as the fallible fallback so the typed `Pid<A>` /
/// `Name<M>` paths stay the obvious default and an agentic caller reaches for a
/// present primitive instead of inventing a workaround. Liveness is identical
/// to [`send_to`]: identity-bound, no redirect, [`SendError::Dead`] once the
/// addressed incarnation is gone. Panics if called outside `Runtime::run()`.
/// Because the message type is not checked at compile time here, this is the
/// one send that can genuinely be live-but-wrong: the actor may be alive yet
/// expose no channel for `M`, in which case you get [`SendError::NoChannel`]
/// back instead of a misdelivery. Liveness and redirect behavior are
/// otherwise identical to [`send_to`]: identity-bound, no redirect,
/// [`SendError::Dead`] once the addressed incarnation is gone. Prefer
/// `send_to` with a typed `Pid<A>` whenever you have one; reach for this only
/// when you don't. Panics if called outside [`run`](crate::run).
///
/// [`Down`]: crate::Down
pub fn send_dyn<M: Send + 'static>(pid: Pid, msg: M) -> Result<(), SendError<M>> {
+477 -205
View File
@@ -65,6 +65,8 @@
//! word stores are `Release`, loads are `Acquire`. The chain that matters:
//! the park path stores `sp` (Relaxed) *before* its Release transition; any
//! later Acquire transition/load of the word therefore observes that `sp`.
//! RFC 019's `hwm` (and the shrink that reads it) piggybacks this exact
//! pattern in the same pre-Release window and adds no edges.
//! The run-queue mutex independently provides the same edges today; the
//! word's own ordering is what phase 3's lock-free queue will rely on.
//!
@@ -86,21 +88,28 @@
//! # Termination (counter-based)
//!
//! The old all-clear scanned the slot table under the big lock. Now:
//! exit when `io_out == 0` (read *before* the queue lock, phase-1 ordering)
//! and, under the queue lock, the queue is empty and `live_actors == 0`.
//! `live_actors` is incremented in `spawn` before the enqueue and decremented
//! at the very END of `finalize_actor`, strictly after every wakeup that
//! finalize produces has been enqueued. The soundness crux: any enqueue
//! targets a live (not-yet-finalized) actor, so `live == 0` implies no wakeup
//! can still be in flight; combined with "spawner is itself live", observing
//! `(queue empty, live == 0)` under the queue lock means no work can ever
//! appear again.
//! exit when `io_outstanding + io_fd_waiters == 0` (two Relaxed/Acquire
//! atomic loads, read *before* the queue pop) and, under the queue lock,
//! the queue is empty and `live_actors == 0`. `live_actors` is incremented
//! in `spawn` before the enqueue and decremented at the very END of
//! `finalize_actor`, strictly after every wakeup that finalize produces has
//! been enqueued. The soundness crux: any enqueue targets a live
//! (not-yet-finalized) actor, so `live == 0` implies no wakeup can still be
//! in flight; combined with "spawner is itself live", observing
//! `(queue empty, live == 0)` means no work can ever appear again.
//!
//! # Timer / IO drain (try-lock, one-winner)
//! # Scheduler park/wake (RFC 018)
//!
//! Unchanged from phase 1: one winner per round drains due timers and IO
//! completions from their own mutexes; wakeups go through the unpark
//! protocol like everyone else's.
//! Schedulers sleep on per-thread futex parkers via the coordination layer
//! (`park.rs`), NOT on a shared wake pipe. IO backends are producers behind
//! a two-call contract — make the actor runnable (`unpark_at`), whose
//! `enqueue` tail wakes exactly one parked scheduler. The blocking pool and
//! epoll thread each route their own completions (driver-enqueues); there
//! is no shared completion queue, no drain lock, no one-winner drain phase.
//! Timers fire two ways: a busy-path due-check every loop iteration (one
//! Relaxed load of the earliest-deadline snapshot when no timer is armed),
//! and the timekeeper — at most one parked scheduler holds the timer
//! deadline, so an expiry wakes one scheduler, not a herd.
use crate::actor::{
clear_current_pid, is_actor_done, reset_actor_done, set_current_actor_box,
@@ -153,6 +162,8 @@ pub struct Config {
alloc_interval: u32,
timeslice_cycles: u64,
stack_pool_cap: usize,
stack_reserve: usize,
stack_guard: usize,
max_actors: usize,
wake_slot: bool,
node_id: crate::pg::NodeId,
@@ -168,6 +179,8 @@ impl Config {
alloc_interval: crate::preempt::DEFAULT_ALLOC_INTERVAL,
timeslice_cycles: crate::preempt::DEFAULT_TIMESLICE_CYCLES,
stack_pool_cap: n * 4,
stack_reserve: DEFAULT_STACK_RESERVE,
stack_guard: DEFAULT_STACK_GUARD,
max_actors: DEFAULT_MAX_ACTORS,
wake_slot: false,
node_id: crate::pg::DEFAULT_NODE_ID,
@@ -187,6 +200,8 @@ impl Config {
alloc_interval: crate::preempt::DEFAULT_ALLOC_INTERVAL,
timeslice_cycles: crate::preempt::DEFAULT_TIMESLICE_CYCLES,
stack_pool_cap: max * 4,
stack_reserve: DEFAULT_STACK_RESERVE,
stack_guard: DEFAULT_STACK_GUARD,
max_actors: DEFAULT_MAX_ACTORS,
wake_slot: false,
node_id: crate::pg::DEFAULT_NODE_ID,
@@ -219,6 +234,31 @@ impl Config {
self
}
/// Default per-actor stack reserve (RFC 019). A *virtual* reservation —
/// anonymous mmap is demand-paged, so RSS follows touched pages, not
/// this number — but overflowing it hits the guard and dies. Page-rounded.
/// Per-actor override: `SpawnOpts::stack_reserve`.
/// Default: [`DEFAULT_STACK_RESERVE`] (64 KiB) — the million-cheap-actors
/// story is unchanged; big stacks are opt-in.
pub fn stack_reserve(mut self, n: usize) -> Self {
assert!(n > 0, "stack_reserve must be non-zero");
self.stack_reserve = n;
self
}
/// Default PROT_NONE guard below each stack (RFC 019). Address space
/// only. Page-rounded. Rust overflow is caught by any single page
/// (probestack touches pages in order); the wide default exists for
/// unprobed FFI frames, which can step over a small guard in one
/// `sub rsp`. Per-actor override: `SpawnOpts::guard_size`.
/// Default: [`DEFAULT_STACK_GUARD`] (1 MiB — the kernel's
/// `stack_guard_gap` convention; see its doc for why width is free).
pub fn stack_guard(mut self, n: usize) -> Self {
assert!(n > 0, "stack_guard must be non-zero");
self.stack_guard = n;
self
}
/// Capacity of the actor slot table — the maximum number of
/// **simultaneously live** actors (total spawned over a run is unbounded;
/// slots are recycled). The table is a fixed slab allocated once at
@@ -286,6 +326,8 @@ impl Default for Config {
alloc_interval: crate::preempt::DEFAULT_ALLOC_INTERVAL,
timeslice_cycles: crate::preempt::DEFAULT_TIMESLICE_CYCLES,
stack_pool_cap: avail * 4,
stack_reserve: DEFAULT_STACK_RESERVE,
stack_guard: DEFAULT_STACK_GUARD,
max_actors: DEFAULT_MAX_ACTORS,
wake_slot: false,
node_id: crate::pg::DEFAULT_NODE_ID,
@@ -376,7 +418,48 @@ impl RuntimeStats {
// Slot — packed state word + hot atomics + cold lifecycle data
// ---------------------------------------------------------------------------
pub(crate) const ACTOR_STACK_SIZE: usize = 64 * 1024;
/// Default usable stack reserve per actor (RFC 019). See [`Config::stack_reserve`].
pub const DEFAULT_STACK_RESERVE: usize = 64 * 1024;
/// Default PROT_NONE guard below each actor stack (RFC 019). Raised from one
/// page so unprobed C frames cannot leap it. See [`Config::stack_guard`].
///
/// 1 MiB, following the kernel's own answer to the same problem: after Stack
/// Clash (2017) the main-thread guard gap became `stack_guard_gap` = 256
/// pages, because 4 KiB was jumpable by one honest `sub rsp` and no small
/// constant was defensible. Guard pages are PROT_NONE: virtual address space
/// only — zero RSS, zero page-table entries, no overcommit charge — so the
/// wide default is free at any actor count (1 M actors ≈ 1 TiB of VA against
/// a 128 TiB budget). A frame that jumps even this lands in the tier-2
/// overshoot window of the SIGSEGV diagnostic (`signal.rs`) instead of
/// silence.
pub const DEFAULT_STACK_GUARD: usize = 1024 * 1024;
/// RFC 019 §3: minimum releasable span (`sp − hwm` at park) before the
/// park-path shrink spends a syscall. A constant, not a `Config` field
/// (ratified): nobody tunes this well and the measured stakes are low — a
/// threshold-sized `MADV_FREE` costs ~3 µs against a ~100 ns park, paid
/// only on spike-recovery parks, which are rare by construction and *were*
/// the spike. Steady-state actors never reach the syscall: their check is
/// two Relaxed loads and a compare on a line the context-save just wrote.
pub const SHRINK_THRESHOLD: usize = 256 * 1024;
/// RFC 019 §3: parks between shrinks of one actor. Guards a few-µs cost, so
/// it can be coarse (parks, not wall time); the kernel's
/// reclaim-under-pressure-only handling of `MADV_FREE` is the real release
/// hysteresis — re-touched-before-pressure pages cost a 0.24 µs/page
/// cancel-write and no fault. A constant, not `Config` (ratified, same
/// rationale as [`SHRINK_THRESHOLD`]).
pub const SHRINK_COOLDOWN: u32 = 64;
/// RFC 019 §6: the entry-end span (highest addresses — the frames the next
/// actor faults first) a recycled stack keeps resident; everything below it
/// is `MADV_DONTNEED`ed before the stack re-enters the pool. Ratified as a
/// constant, not Config, alongside the shrink knobs; the 64 KiB value was a
/// flagged Claude-solo call at ratification — it equals the default reserve,
/// so with an unraised Config the zap is a no-op and only Configs that raise
/// the default reserve pay it.
pub const RECYCLE_RETAIN: usize = 64 * 1024;
pub(crate) type Closure = Box<dyn FnOnce() + Send>;
@@ -419,6 +502,35 @@ pub(crate) struct Slot {
/// Release transition out of Running; read after the Acquire transition
/// Queued→Running. Relaxed is sufficient — ordering rides on `word`.
sp: AtomicUsize,
/// RFC 019: sampled stack high-water — the minimum `sp` ever stored above,
/// i.e. the deepest excursion *observed at a switch point*. Advisory:
/// correctness never depends on it; its one job is "is a shrink worth a
/// syscall?". Declared adjacent to `sp` so the min-update dirties the
/// line the context-save just wrote. Same single-writer Relaxed
/// discipline as `sp`; reset to the fresh `sp` at install.
hwm: AtomicUsize,
/// RFC 019: parks since the last shrink (or install). Counted on every
/// pass through the Park arm by the owning scheduler thread; the shrink
/// fires only once this clears [`SHRINK_COOLDOWN`] *and* the releasable
/// span clears [`SHRINK_THRESHOLD`]. Single-writer Relaxed.
parks_since_shrink: AtomicU32,
/// RFC 019: shrinks performed on this incarnation (introspection lands
/// with the RFC's introspect surface; the counter exists from birth so
/// tests can rely on install resetting it). Single-writer Relaxed.
shrink_count: AtomicU32,
/// RFC 019 §7 — stack geometry for the SIGSEGV classifier, readable
/// without the cold lock (the `Stack` itself lives under it). Written in
/// `install_actor` before the Release publish; consulted by the handler
/// only while `preempt::CURRENT_SLOT` points here, i.e. while this actor
/// is on-CPU, so the values are never stale where they are read. 0 =
/// never installed. Usable top of the stack.
pub(crate) diag_stack_top: AtomicUsize,
/// See `diag_stack_top`: the reserve (usable) size.
pub(crate) diag_stack_reserve: AtomicUsize,
/// See `diag_stack_top`: the guard size.
pub(crate) diag_stack_guard: AtomicUsize,
/// See `diag_stack_top`: `(idx << 32) | generation`, for the message.
pub(crate) diag_pid: AtomicU64,
/// Pointer into the actor's `Arc<AtomicBool>` stop flag. Set at spawn,
/// nulled at finalize. The box outlives every read: it is only ever read
/// on the resume path while the actor cannot be finalized (it is on-CPU).
@@ -490,6 +602,13 @@ impl Slot {
Self {
word: StateWord::new(),
sp: AtomicUsize::new(0),
hwm: AtomicUsize::new(0),
parks_since_shrink: AtomicU32::new(0),
shrink_count: AtomicU32::new(0),
diag_stack_top: AtomicUsize::new(0),
diag_stack_reserve: AtomicUsize::new(0),
diag_stack_guard: AtomicUsize::new(0),
diag_pid: AtomicU64::new(0),
stop_ptr: AtomicPtr::new(std::ptr::null_mut()),
closure: AtomicPtr::new(std::ptr::null_mut()),
overruns: AtomicU64::new(0),
@@ -543,6 +662,23 @@ impl Slot {
/// Read the overrun tally (Relaxed; the snapshot reads cross-thread).
#[inline]
/// RFC 019 §8 — the stack introspection tuple, all lock-free:
/// `(reserve, guard, top, hwm, parks_since_shrink, shrink_count)`.
/// Geometry from the c6 diag atomics (install-time, gen-coherent under
/// `read_slot`'s gen check exactly like the other counters); `hwm` is the
/// §2 sampled high-water (lowest saved sp). All zeros before first
/// install.
pub(crate) fn stack_introspect(&self) -> (usize, usize, usize, usize, u32, u32) {
(
self.diag_stack_reserve.load(Ordering::Relaxed),
self.diag_stack_guard.load(Ordering::Relaxed),
self.diag_stack_top.load(Ordering::Relaxed),
self.hwm.load(Ordering::Relaxed),
self.parks_since_shrink.load(Ordering::Relaxed),
self.shrink_count.load(Ordering::Relaxed),
)
}
pub(crate) fn overruns(&self) -> u64 {
self.overruns.load(Ordering::Relaxed)
}
@@ -750,8 +886,21 @@ pub(crate) struct RuntimeInner {
pub(crate) io: Mutex<Option<IoThread>>,
/// Monotonic `MonitorId` source. Never reused.
pub(crate) next_monitor_id: AtomicU64,
/// Try-lock: exactly one scheduler thread drains timers/IO per iteration.
drain_lock: Mutex<()>,
/// RFC 018: the scheduler coordination layer — per-scheduler parkers,
/// idle mask, wake protocol, timekeeper role, earliest-deadline
/// snapshot. Arc'd because `Timers` shares it (insert-side deadline
/// notes run under the timers mutex).
pub(crate) coord: Arc<crate::park::Coordinator>,
/// `block_on_io` requests in flight. Incremented by the submitter
/// BEFORE submit (underflow-proof), decremented by the pool thread on
/// completion. Read lock-free by the idle path's termination verdict —
/// the per-pop `io.lock` of the drain era is gone.
pub(crate) io_outstanding: AtomicU32,
/// Parked fd waiters. Incremented by the registrar BEFORE
/// `epoll_register` (rolled back on error), decremented by whoever
/// consumes the registration (epoll thread on readiness, canceller on
/// an unwound wait). Same lock-free verdict read as `io_outstanding`.
pub(crate) io_fd_waiters: AtomicU32,
/// Per-thread stats, indexed by scheduler thread slot (0..N).
pub(crate) stats: Vec<SchedulerStats>,
/// Global counters for RFC 000 primitives.
@@ -782,17 +931,23 @@ pub(crate) struct RuntimeInner {
pub(crate) stack_pool: RawMutex<Vec<crate::stack::Stack>>,
/// Maximum number of stacks to retain in the pool.
pub(crate) stack_pool_cap: usize,
/// Default stack shape (RFC 019), pre-page-rounded so it compares exactly
/// against `Stack::shape()`. Only stacks of exactly this shape are pooled.
pub(crate) stack_reserve: usize,
pub(crate) stack_guard: usize,
}
impl RuntimeInner {
// Private constructor taking the parsed Config fields one-for-one; a params
// struct would only move the same 8 values across the call boundary.
// struct would only move the same 10 values across the call boundary.
#[allow(clippy::too_many_arguments)]
fn new(
thread_count: usize,
alloc_interval: u32,
timeslice_cycles: u64,
stack_pool_cap: usize,
stack_reserve: usize,
stack_guard: usize,
max_actors: usize,
wake_slot: bool,
node_id: crate::pg::NodeId,
@@ -802,6 +957,12 @@ impl RuntimeInner {
let slots: Box<[Slot]> = (0..max_actors).map(|_| Slot::vacant()).collect();
// Low indices on top of the stack so early spawns get low pids.
let free: Vec<u32> = (0..max_actors as u32).rev().collect();
// RFC 018: the coordination layer (asserts thread_count <= 64), and
// the timers' hook into it — every insert under the timers mutex
// notes its deadline (busy-path snapshot + timekeeper re-arm).
let coord = Arc::new(crate::park::Coordinator::new(thread_count));
let mut timers = Timers::new();
timers.attach_coordinator(coord.clone());
Arc::new(Self {
run_queue: crate::run_queue::RunQueue::new(thread_count, max_actors),
slots,
@@ -810,10 +971,12 @@ impl RuntimeInner {
root_bits: AtomicU64::new(u64::MAX),
root_exited: AtomicBool::new(false),
root_swept: AtomicBool::new(false),
timers: Mutex::new(Timers::new()),
timers: Mutex::new(timers),
io: Mutex::new(None),
next_monitor_id: AtomicU64::new(0),
drain_lock: Mutex::new(()),
coord,
io_outstanding: AtomicU32::new(0),
io_fd_waiters: AtomicU32::new(0),
stats,
io_parked: AtomicU32::new(0),
sleeping: AtomicU32::new(0),
@@ -826,6 +989,8 @@ impl RuntimeInner {
process_groups: RawMutex::new(crate::pg::ProcessGroups::new()),
stack_pool: RawMutex::new(Vec::new()),
stack_pool_cap,
stack_reserve: crate::stack::round_to_pages(stack_reserve),
stack_guard: crate::stack::round_to_pages(stack_guard),
})
}
@@ -874,6 +1039,13 @@ impl RuntimeInner {
);
self.run_queue.push(pid);
crate::te!(crate::trace::Event::Enqueue(pid));
// RFC 018 enqueue wake (fixes the silent enqueue): if a scheduler
// is parked, wake exactly one. The fast path when everyone is busy
// is a fence + one Relaxed load of an unmodified line — the
// pure-compute hot path pays (almost) nothing. Bias is over-wake:
// a spurious wake costs one futex round-trip and a failed pop; a
// missed wake would cost a stranded actor.
self.coord.wake_one_if_idle();
}
/// Make `pid` runnable if it is parked; coalesce or defer otherwise.
@@ -1010,6 +1182,9 @@ pub struct Runtime {
/// Initialise the runtime with the given config. Returns a reusable handle.
pub fn init(config: Config) -> Runtime {
// RFC 019 §7: one process-global SIGSEGV handler, installed before any
// scheduler thread (and so before any classifiable fault) can exist.
crate::signal::install_once();
let n = config.resolved_thread_count();
Runtime {
inner: RuntimeInner::new(
@@ -1017,6 +1192,8 @@ pub fn init(config: Config) -> Runtime {
config.alloc_interval,
config.timeslice_cycles,
config.stack_pool_cap,
config.stack_reserve,
config.stack_guard,
config.max_actors,
config.wake_slot,
config.node_id,
@@ -1076,7 +1253,16 @@ impl Runtime {
self.inner.live_actors.load(Ordering::Acquire), 0,
"run() called while previous run still active"
);
let io_thread = match IoThread::start() {
// RFC 018: the IO producers reach the runtime (slot table + unpark)
// through a Weak, so no RuntimeInner → IoThread → RuntimeInner cycle
// forms. Reset the in-flight counters BEFORE the threads can touch
// them (a prior run left them at 0 on a clean exit; the asserts pin
// that).
debug_assert_eq!(self.inner.io_outstanding.load(Ordering::Acquire), 0);
debug_assert_eq!(self.inner.io_fd_waiters.load(Ordering::Acquire), 0);
self.inner.io_outstanding.store(0, Ordering::Release);
self.inner.io_fd_waiters.store(0, Ordering::Release);
let io_thread = match IoThread::start(Arc::downgrade(&self.inner)) {
Ok(io) => io,
Err(e) => panic!("failed to start IO thread: {e}"),
};
@@ -1179,6 +1365,8 @@ impl Runtime {
}
self.inner.io_parked.store(0, Ordering::Relaxed);
self.inner.sleeping.store(0, Ordering::Relaxed);
self.inner.io_outstanding.store(0, Ordering::Relaxed);
self.inner.io_fd_waiters.store(0, Ordering::Relaxed);
RUNTIME.with(|r| *r.borrow_mut() = None);
@@ -1263,6 +1451,99 @@ pub const ROOT_PID: Pid = Pid::new(u32::MAX, u32::MAX);
// Spawn-side slot installation
// ---------------------------------------------------------------------------
// ---------------------------------------------------------------------------
// Stack shrink — RFC 019 §3 (park path only)
// ---------------------------------------------------------------------------
/// The per-park shrink check. Called from the `YieldIntent::Park` arm inside
/// the owned window (see the assert-comment at the call site). Fast path —
/// no spike since the last shrink — is two Relaxed loads, a compare, and the
/// park counter bump, all on the slot line the context-save just wrote.
///
/// On a shrink: `MADV_FREE` the whole pages of `[hwm, sp − redzone)` (the
/// inward-rounded range from [`crate::stack::shrink_range`]), then reset
/// `hwm = sp` and the park counter. MADV_FREE only *marks*: the kernel
/// reclaims under pressure, skips re-dirtied pages, and refaults zero pages
/// for writes after reclaim — so an over-eager mark costs a cancel-write,
/// never data.
fn maybe_shrink_stack(slot: &Slot) {
let parks = slot.parks_since_shrink.load(Ordering::Relaxed).saturating_add(1);
slot.parks_since_shrink.store(parks, Ordering::Relaxed);
let sp = slot.sp.load(Ordering::Relaxed);
let hwm = slot.hwm.load(Ordering::Relaxed);
if sp.wrapping_sub(hwm) < SHRINK_THRESHOLD || sp < hwm {
return; // common case: nothing worth a syscall
}
if parks < SHRINK_COOLDOWN {
return;
}
let page = crate::stack::page_size();
if let Some((addr, len)) = crate::stack::shrink_range(hwm, sp, page) {
// Advisory: on the (kernel-config) chance MADV_FREE is unsupported,
// failing silently degrades to "never shrinks", which is correct.
unsafe {
libc::madvise(addr as *mut libc::c_void, len, libc::MADV_FREE);
}
slot.hwm.store(sp, Ordering::Relaxed);
slot.parks_since_shrink.store(0, Ordering::Relaxed);
slot.shrink_count.store(
slot.shrink_count.load(Ordering::Relaxed).saturating_add(1),
Ordering::Relaxed,
);
}
}
// ---------------------------------------------------------------------------
// Stack acquisition / recycling — RFC 019 pool rule
// ---------------------------------------------------------------------------
/// Get a stack of the shape `opts` requests (`None` fields ⇒ the runtime
/// defaults).
///
/// Pool rule (RFC 019 §1): the pool is a uniform `Vec<Stack>` of
/// default-shaped stacks and stays that way. Default-shaped requests try the
/// pool first; custom shapes always mmap fresh (and `recycle_stack` never
/// admits them, so a pooled stack is default-shaped by induction). The pool
/// lock is dropped before any mmap: no syscall ever stalls another spawner.
pub(crate) fn acquire_stack(
inner: &RuntimeInner,
opts: crate::scheduler::SpawnOpts,
) -> crate::stack::Stack {
let reserve = opts.stack_reserve.unwrap_or(inner.stack_reserve);
let guard = opts.guard_size.unwrap_or(inner.stack_guard);
let default_shaped = crate::stack::round_to_pages(reserve) == inner.stack_reserve
&& crate::stack::round_to_pages(guard) == inner.stack_guard;
if default_shaped {
if let Some(stack) = inner.stack_pool.lock().pop() {
return stack;
}
}
match crate::stack::Stack::new(reserve, guard) {
Ok(stack) => stack,
Err(e) => panic!("stack allocation failed: {e}"),
}
}
/// Return a dead actor's stack: pooled if default-shaped and under cap,
/// otherwise dropped here → munmap (custom shapes and cap overflow alike).
pub(crate) fn recycle_stack(inner: &RuntimeInner, stack: crate::stack::Stack) {
if stack.shape() == (inner.stack_reserve, inner.stack_guard) {
// RFC 019 §6: zap the dead spike before pooling, BEFORE taking the
// pool lock — acquire_stack's invariant is that no syscall ever
// stalls another spawner under it. On the rare cap-overflow the zap
// is wasted work ahead of the munmap; harmless, and cheaper than a
// second lock round-trip to find out.
stack.recycle_zap(RECYCLE_RETAIN);
let mut pool = inner.stack_pool.lock();
if pool.len() < inner.stack_pool_cap {
pool.push(stack);
}
// else: fall through — drop → munmap.
}
// Custom-shaped (or cap overflow): `stack` drops here → munmap.
}
/// Install a freshly spawned actor into the slot `idx` (which must have come
/// from `allocate_slot`) and publish it as Queued. Returns the new `Pid`.
/// Called by `scheduler::spawn_under`; lives here next to its inverse
@@ -1280,6 +1561,11 @@ pub(crate) fn install_actor(
let pid = Pid::new(idx, gen);
let stop = Arc::new(AtomicBool::new(false));
// RFC 019 §7: geometry for the SIGSEGV classifier, captured before the
// Stack moves under the cold lock. Ordered before readers by the
// publish below.
let (diag_reserve, diag_guard) = stack.shape();
let diag_top = stack.top() as usize;
slot.stop_ptr.store(Arc::as_ptr(&stop) as *mut _, Ordering::Release);
{
let mut cold = slot.cold.lock();
@@ -1291,6 +1577,15 @@ pub(crate) fn install_actor(
cold.pending_io_result = None;
}
slot.sp.store(sp, Ordering::Relaxed);
// RFC 019: a fresh incarnation starts with its high-water at the fresh
// top-of-stack `sp` and its shrink bookkeeping zeroed.
slot.hwm.store(sp, Ordering::Relaxed);
slot.parks_since_shrink.store(0, Ordering::Relaxed);
slot.shrink_count.store(0, Ordering::Relaxed);
slot.diag_stack_top.store(diag_top, Ordering::Relaxed);
slot.diag_stack_reserve.store(diag_reserve, Ordering::Relaxed);
slot.diag_stack_guard.store(diag_guard, Ordering::Relaxed);
slot.diag_pid.store(((idx as u64) << 32) | gen as u64, Ordering::Relaxed);
slot.store_closure(closure);
slot.reset_counters();
inner.live_actors.fetch_add(1, Ordering::Relaxed);
@@ -1391,13 +1686,7 @@ fn finalize_actor(inner: &Arc<RuntimeInner>, pid: Pid, outcome: Outcome) {
// (the trap sender can unpark its receiver — keep that outside too).
let supervisor_pid = actor.supervisor;
let Actor { stack, .. } = actor;
{
let mut pool = inner.stack_pool.lock();
if pool.len() < inner.stack_pool_cap {
pool.push(stack);
}
// else: drop here → munmap, same as before
}
recycle_stack(inner, stack);
// Deliver to supervisor. ROOT_PID resolves to no slot → silently absorbed.
let sender = inner.slot_at(supervisor_pid).and_then(|sup| {
@@ -1493,130 +1782,76 @@ fn stop_live_actors(inner: &Arc<RuntimeInner>) {
}
}
// ---------------------------------------------------------------------------
// Timer firing — shared by the busy-path due-check and the timekeeper
// ---------------------------------------------------------------------------
/// Pop and dispatch every due timer. `pop_due` re-anchors the
/// earliest-deadline snapshot under the timers mutex before returning, so
/// a caller that raced a concurrent insert simply comes back on the next
/// due-check. Dispatch runs with the timers lock released.
fn fire_due_timers(inner: &Arc<RuntimeInner>, try_only: bool) {
let due = if try_only {
// Busy path: if another scheduler is already in the timers mutex
// (firing, inserting, or peeking) skip — the snapshot stays due
// until someone actually pops, so the check re-fires next loop.
match inner.timers.try_lock() {
Ok(mut t) => t.pop_due(std::time::Instant::now()),
Err(std::sync::TryLockError::WouldBlock) => return,
Err(std::sync::TryLockError::Poisoned(e)) => {
panic!("smarm: timers lock poisoned (core corrupt): {e}")
}
}
} else {
match inner.timers.lock() {
Ok(mut t) => t.pop_due(std::time::Instant::now()),
Err(e) => panic!("smarm: timers lock poisoned (core corrupt): {e}"),
}
};
for entry in due {
match entry.reason {
// A sleep expiry is just an unpark: the protocol handles
// every interleaving — Parked (re-queue), Running (the
// actor is between `timers.insert_sleep` and
// `park_current`; RunningNotified makes the upcoming park
// re-queue), or gone (no-op).
crate::timer::Reason::Sleep { epoch } => inner.unpark_at(entry.pid, epoch),
crate::timer::Reason::WaitTimeout { target, epoch } => {
// The callback may call unpark_at itself.
target.on_timeout(entry.pid, epoch);
}
// A `send_after` deadline: run the captured delivery thunk.
// It resolves the destination through the registry and
// sends now (a send can unpark a receiver) — same as any
// other in-loop unpark. The timers lock is already
// released; lock order Leaf -> Channel is preserved by the
// send itself. `pop_due` only returns still-armed Sends, so
// a cancelled one never reaches here.
crate::timer::Reason::Send { fire } => fire(),
}
}
}
// ---------------------------------------------------------------------------
// schedule_loop — runs on each scheduler OS thread
// ---------------------------------------------------------------------------
fn schedule_loop(inner: &Arc<RuntimeInner>, slot_idx: usize) {
// RFC 019 §7: a guard hit leaves no stack to handle the signal on.
crate::signal::register_altstack();
crate::preempt::configure_preempt(inner.alloc_interval, inner.timeslice_cycles);
let stats = &inner.stats[slot_idx];
loop {
// ----------------------------------------------------------------
// 1. Try to win the drain lock (timers + IO). One winner per round;
// losers skip immediately and proceed to step 2.
// 1. Busy-path timer due-check (RFC 018 design point (a)): under
// saturation nobody parks, so no timekeeper exists — due timers
// must still fire. One Relaxed load + branch when no timer is
// armed; the clock is read only when one is.
// ----------------------------------------------------------------
if let Ok(_drain_guard) = inner.drain_lock.try_lock() {
// Timers and IO live behind their own mutexes (phase 1), so the
// pure-yield / pure-compute hot path never contends a global lock
// just to discover there is nothing to drain. The clock is read
// only when the timer heap is non-empty.
let due = {
let mut t = match inner.timers.lock() {
Ok(t) => t,
Err(e) => panic!("smarm: timers lock poisoned (core corrupt): {e}"),
};
if t.is_empty() {
Vec::new()
} else {
t.pop_due(std::time::Instant::now())
}
};
let completions = match inner.io.lock() {
Ok(mut io) => io
.as_mut()
.map(|io| {
// Consume wake-pipe bytes ONLY here, under the drain
// lock and strictly before draining completions.
// Producers push their completion before writing the
// byte, so every byte consumed here has its completion
// visible to the drain below. Consuming bytes anywhere
// else — in particular after an idle poll, outside the
// lock — loses wakeups: a try_lock loser can eat the
// byte for a completion the winner never saw, leaving
// it stranded (and its EPOLLONESHOT fd disarmed) until
// an unrelated timer forces another drain pass.
crate::io::drain_wake_pipe(io.wake_fd());
io.drain_completions()
})
.unwrap_or_default(),
Err(e) => panic!("smarm: io lock poisoned (core corrupt): {e}"),
};
for entry in due {
match entry.reason {
// A sleep expiry is just an unpark: the protocol handles
// every interleaving — Parked (re-queue), Running (the
// actor is between `timers.insert_sleep` and
// `park_current`; RunningNotified makes the upcoming park
// re-queue), or gone (no-op).
crate::timer::Reason::Sleep { epoch } => {
inner.unpark_at(entry.pid, epoch)
}
crate::timer::Reason::WaitTimeout { target, epoch } => {
// The callback may call unpark_at itself.
target.on_timeout(entry.pid, epoch);
}
// A `send_after` deadline: run the captured delivery thunk.
// It resolves the destination through the registry and
// sends now (a send can unpark a receiver) — same as any
// other in-loop unpark. The timers lock is already
// released; lock order Leaf -> Channel is preserved by the
// send itself. `pop_due` only returns still-armed Sends, so
// a cancelled one never reaches here.
crate::timer::Reason::Send { fire } => fire(),
}
}
for completion in completions {
match completion {
crate::io::Completion::Blocking { pid, epoch, result } => {
match inner.io.lock() {
Ok(mut io) => {
if let Some(io) = io.as_mut() {
io.outstanding = io.outstanding.saturating_sub(1);
}
}
Err(e) => {
panic!("smarm: io lock poisoned (core corrupt): {e}")
}
}
// Stash the result under the cold lock, then unpark.
// The protocol also covers the submit→park window
// (RunningNotified), which the old code missed for
// Blocking completions — a latent lost wakeup.
if let Some(slot) = inner.slot_at(pid) {
{
let mut cold = slot.cold.lock();
if slot.generation() == pid.generation() {
cold.pending_io_result = Some(result);
} else {
// Actor died (stopped) with the op in
// flight; discard the result.
}
}
inner.unpark_at(pid, epoch);
}
}
crate::io::Completion::FdReady { fd, events: _ } => {
// Resolve the parked pid under the io lock, then wake
// through the protocol. Lock order: io before all.
let parked = match inner.io.lock() {
Ok(mut io) => io.as_mut().and_then(|io| {
let entry = io.waiters.remove(&fd);
io.epoll_deregister(fd);
entry
}),
Err(e) => {
panic!("smarm: io lock poisoned (core corrupt): {e}")
}
};
if let Some((pid, epoch)) = parked {
inner.unpark_at(pid, epoch);
}
}
}
}
} // drain_guard drops here
if inner.coord.deadline_due() {
fire_due_timers(inner, true);
}
// ----------------------------------------------------------------
// 2. Pop a runnable pid. Pop order (RFC 005): wake slot first, then
@@ -1625,7 +1860,7 @@ fn schedule_loop(inner: &Arc<RuntimeInner>, slot_idx: usize) {
// ----------------------------------------------------------------
enum Pop {
Got(Pid),
Idle { io_outstanding: u32, wake_fd: Option<std::os::fd::RawFd> },
Idle,
AllDone,
/// Root has exited and nothing is runnable: stop the parked-forever
/// remainder, then re-pop. Fires at most once per run.
@@ -1650,19 +1885,12 @@ fn schedule_loop(inner: &Arc<RuntimeInner>, slot_idx: usize) {
crate::te!(crate::trace::Event::SlotPop(pid));
pid
} else {
// Read IO liveness BEFORE the queue lock (phase-1 ordering: a
// completion resurrects an actor only via the drain path, whose
// enqueue would be visible under the queue lock we take next).
let (io_out, io_fd) = {
let io = match inner.io.lock() {
Ok(io) => io,
Err(e) => panic!("smarm: io lock poisoned (core corrupt): {e}"),
};
match io.as_ref() {
Some(io) => (io.outstanding + io.waiters.len() as u32, Some(io.wake_fd())),
None => (0, None),
}
};
// Read IO liveness BEFORE the queue pop — two atomic loads now
// (RFC 018), not a per-pop `io.lock`: a completion resurrects
// an actor via the producer's own unpark→enqueue, whose entry
// would be visible to the pop below.
let io_out = inner.io_outstanding.load(Ordering::Acquire)
+ inner.io_fd_waiters.load(Ordering::Acquire);
stats.run_queue_len.store(inner.run_queue.len(), Ordering::Relaxed);
let pop = match inner.run_queue.pop() {
@@ -1694,7 +1922,7 @@ fn schedule_loop(inner: &Arc<RuntimeInner>, slot_idx: usize) {
// the idle wait below on the next pass.
Pop::RootDrain
} else {
Pop::Idle { io_outstanding: io_out, wake_fd: io_fd }
Pop::Idle
}
}
};
@@ -1709,22 +1937,15 @@ fn schedule_loop(inner: &Arc<RuntimeInner>, slot_idx: usize) {
Ok(mut timers) => timers.clear(),
Err(e) => panic!("smarm: timers lock poisoned (core corrupt): {e}"),
}
// Terminal wake: a sibling scheduler may be blocked in its
// idle wait on a snapshot that is now terminally stale — an
// orphaned long deadline (it would sleep it out in full) or
// a stale `io_outstanding > 0` from a stop-cancelled waiter
// (it would block in poll(-1) forever; cancellation produces
// no completion, so nothing else writes the wake pipe).
// One byte wakes every poller; each re-runs the verdict,
// reaches AllDone itself, and re-wakes — idempotent.
match inner.io.lock() {
Ok(io) => {
if let Some(io) = io.as_ref() {
io.wake();
}
}
Err(e) => panic!("smarm: io lock poisoned (core corrupt): {e}"),
}
// Terminal wake (replaces the wake-pipe byte): a sibling
// may be parked on a snapshot that is now terminally
// stale — an orphaned long deadline, or a stale
// `io_fd_waiters > 0` from a stop-cancelled waiter
// (cancellation produces no completion, so nothing else
// will ever wake it). `wake_all` permits every parker;
// each sibling re-runs the verdict, reaches AllDone
// itself, and re-wakes — idempotent.
inner.coord.wake_all();
return;
}
Pop::RootDrain => {
@@ -1734,40 +1955,51 @@ fn schedule_loop(inner: &Arc<RuntimeInner>, slot_idx: usize) {
stop_live_actors(inner);
continue;
}
Pop::Idle { io_outstanding, wake_fd } => {
// Something is still in flight. Sleep on the appropriate
// source to avoid hammering the queue mutex; retry on wake.
let next_deadline = match inner.timers.lock() {
Ok(timers) => timers.peek_deadline(),
Err(e) => panic!("smarm: timers lock poisoned (core corrupt): {e}"),
};
match (next_deadline, wake_fd) {
(Some(deadline), fd_opt) => {
let now = std::time::Instant::now();
if deadline > now {
let timeout = deadline - now;
match fd_opt {
Some(fd) => {
// Wake only; the byte (if any) is
// consumed by the next drain-lock
// winner in phase 1. Level-triggered
// poll means an unconsumed byte makes
// this return immediately, so a loser
// spins briefly until the winner
// releases — never sleeps through it.
crate::io::poll_wake(fd, Some(timeout));
}
None => thread::sleep(timeout),
}
Pop::Idle => {
// Something is still in flight. Park on our own futex
// until a producer wakes us (enqueue tail), a deadline
// passes, or the re-check finds the world changed.
//
// Timekeeper (RFC 018): at most one parked scheduler
// holds the timer deadline — the first idler to arm it
// parks with a timeout, the rest park indefinitely, so a
// timer expiry wakes one scheduler, not a herd. Peek and
// arm under the timers mutex (the serialization that
// makes the insert-side re-arm race-free).
let tk_deadline = {
let timers = match inner.timers.lock() {
Ok(t) => t,
Err(e) => {
panic!("smarm: timers lock poisoned (core corrupt): {e}")
}
}
(None, Some(fd)) if io_outstanding > 0 => {
// See above: no byte consumption outside phase 1.
crate::io::poll_wake(fd, None);
}
_ => {
thread::sleep(std::time::Duration::from_micros(100));
}
};
timers
.peek_deadline()
.filter(|d| inner.coord.try_arm_timer(slot_idx, *d))
};
// The mandatory post-publish re-check: a producer that
// enqueued (or a verdict input that flipped) before it
// could see our idle bit has left us the evidence.
let _ = inner.coord.park(slot_idx, tk_deadline, || {
!inner.run_queue.is_empty()
|| (inner.live_actors.load(Ordering::Acquire) == 0
&& inner.io_outstanding.load(Ordering::Acquire) == 0
&& inner.io_fd_waiters.load(Ordering::Acquire) == 0)
|| (inner.root_exited.load(Ordering::Acquire)
&& !inner.root_swept.load(Ordering::Acquire))
|| inner.coord.deadline_due()
});
if tk_deadline.is_some() {
// Hand the role back BEFORE firing: pop_due can run
// `Send` thunks that insert new timers, and the
// insert-side re-arm check must see either no
// timekeeper (skip) or a real parked one — never us,
// awake and about to re-peek anyway.
inner.coord.disarm_timer(slot_idx);
// Woken for the deadline, for work, or to re-peek
// after an earlier insert — fire whatever is due;
// the next idle pass re-arms with the new minimum.
fire_due_timers(inner, false);
}
continue;
}
@@ -1780,6 +2012,21 @@ fn schedule_loop(inner: &Arc<RuntimeInner>, slot_idx: usize) {
// by the at-most-once-enqueued invariant nothing else can have
// changed the state of a queued actor.
// ----------------------------------------------------------------
// RFC 018 chain rule: we just took one runnable; if more remain and
// a sibling is parked, wake exactly one so the surplus runs in
// PARALLEL rather than serially behind us (without this the surplus
// is not stranded — we re-pop it after resuming — but it waits out
// our whole timeslice while an idle core sits available). Cheap: the
// queue-length check is queue-local, and `wake_one_if_idle` is a
// fence + one Relaxed mask load when nobody is parked. A Relaxed
// miss here is safe — the enqueue that created the surplus already
// issued its own wake (RFC 018 no-lost-wake); this only sharpens
// parallelism latency.
if !inner.run_queue.is_empty() {
inner.coord.wake_one_if_idle();
}
let slot = match inner.slot_at(pid) {
Some(s) => s,
None => continue, // can't happen for real pids; defensive
@@ -1839,7 +2086,15 @@ fn schedule_loop(inner: &Arc<RuntimeInner>, slot_idx: usize) {
crate::preempt::clear_current_slot();
let intent = YIELD_INTENT.with(|c| c.get());
slot.sp.store(get_actor_sp(), Ordering::Relaxed);
let saved_sp = get_actor_sp();
slot.sp.store(saved_sp, Ordering::Relaxed);
// RFC 019 §2: sampled high-water — one branch + at most one store
// into the line the store above just dirtied. Relaxed and advisory;
// it piggybacks the existing Relaxed-store-before-Release pattern
// (mod docs, "Memory ordering") and adds no edges.
if saved_sp < slot.hwm.load(Ordering::Relaxed) {
slot.hwm.store(saved_sp, Ordering::Relaxed);
}
if is_actor_done() {
crate::te!(crate::trace::Event::Done(pid));
@@ -1861,6 +2116,23 @@ fn schedule_loop(inner: &Arc<RuntimeInner>, slot_idx: usize) {
inner.enqueue(pid);
}
YieldIntent::Park => {
// RFC 019 §3 shrink window (correctness obligation 1):
// this site sits after the `sp` store above and before
// the `park_return` Release transition below publishes
// Parked — the scheduler is on its own stack and the
// actor is saved but not yet stealable, so the madvise
// races nothing (belt). MADV_FREE's cancel-on-write is
// the suspenders: even a racing writer could lose
// nothing written after the mark, and everything below
// live `sp` is dead by definition. Runs on BOTH arms of
// the park_return race — a consumed unpark flag means a
// wasted-but-harmless madvise on a rare window.
//
// This is the ONLY shrink site: the preempt/yield path
// deliberately never checks (§4's bounded leak under
// saturation — syscalls must not fire when scheduler
// cycles are scarcest).
maybe_shrink_stack(slot);
if slot.word.park_return(gen) {
// RFC 007 audit: an in-site park drops its sample
// tail (nothing flushes it; on_resume re-arms).
+443 -196
View File
@@ -1,11 +1,69 @@
//! Scheduler public API — thin façade over the multi-scheduler runtime.
//! Start actors and control them: `run`, `spawn`, `sleep`, timers, and IO waits.
//!
//! All heavy lifting lives in `runtime.rs`. This module exposes the same
//! surface that the rest of the codebase (channel, mutex, io, timer, actor)
//! calls into, plus the public API re-exported from `lib.rs`.
//! ## What is an actor?
//!
//! The single-threaded `run()` entry point is kept as a convenience wrapper
//! around `runtime::init(Config::exact(1)).run(f)`.
//! An actor in smarm is a *green thread*: a lightweight, cooperatively
//! scheduled unit of execution with its own stack, running alongside
//! thousands of others on a handful of OS threads. You give it a closure;
//! smarm gives it a [`Pid`] and a [`JoinHandle`]. Actors talk to each other by
//! sending messages over [`channel`](mod@crate::channel)s, not by sharing memory,
//! so most of the concurrency bugs that come from shared mutable state simply
//! don't arise.
//!
//! Everything in smarm (channels, [`Mutex`](crate::Mutex), timers, IO waits,
//! `gen_server`) needs to run on top of a scheduler. [`run`] starts one and
//! blocks the calling OS thread until the actor you give it finishes; from
//! inside that actor (or any actor it spawns), call [`spawn`] to start more.
//!
//! ```
//! use smarm::{run, spawn};
//!
//! run(|| {
//! let handle = spawn(|| {
//! println!("hello from another actor");
//! });
//! handle.join().unwrap();
//! });
//! ```
//!
//! ## Waiting for an actor: `JoinHandle`
//!
//! [`spawn`] returns a [`JoinHandle`], your only handle on the actor's
//! outcome. Call [`JoinHandle::join`] to block the calling actor until the
//! spawned one finishes:
//!
//! - if it returned normally, `join` returns `Ok(())`;
//! - if it panicked, `join` returns `Err(`[`JoinError`]`)` carrying the panic
//! payload, so a crash in one actor never silently vanishes and never
//! crashes the process; the caller decides what to do with it (log it,
//! propagate it, ignore it).
//!
//! If you don't need the result, drop the `JoinHandle` (or never bind it):
//! the actor keeps running independently. To watch an actor without
//! blocking, or to react to failures across a whole tree of actors, see
//! [`monitor`](mod@crate::monitor), [`link`](mod@crate::link), and
//! [`supervisor`](crate::supervisor) instead.
//!
//! ## Sleeping, timeouts, and IO
//!
//! [`sleep`] parks only the calling actor, not the OS thread underneath it,
//! so thousands of sleeping actors cost nothing but the timer entry.
//! [`send_after`] and [`send_after_named`] schedule a message to be delivered
//! later (Erlang-style `send_after`), and [`cancel_timer`] can call one off
//! before it fires.
//!
//! For blocking or file-descriptor-based IO, see [`block_on_io`],
//! [`wait_readable`], and [`wait_writable`]: they park the calling actor and
//! resume it when the work completes or the fd is ready, again without
//! blocking an OS thread.
//!
//! ## Stopping an actor from the outside
//!
//! [`request_stop`] asks an actor to cooperatively unwind: it's the
//! mechanism `gen_server` shutdown, timeouts, and supervisor restarts are
//! built on. It's best-effort: an actor that never yields, allocates, or
//! blocks (a tight loop with nothing else in it) has no opportunity to
//! notice the request.
use crate::actor::current_pid;
use crate::channel::Sender;
@@ -14,27 +72,21 @@ use crate::runtime::{
self, RuntimeInner, YieldIntent, RUNTIME,
};
use crate::supervisor::Signal;
use std::sync::atomic::Ordering;
use std::sync::Arc;
// ---------------------------------------------------------------------------
// with_runtime / try_with_runtime
// ---------------------------------------------------------------------------
/// Borrow the current runtime. Panics if called outside `Runtime::run()`.
///
/// The whole span runs with preemption disabled. Two reasons, both load-
/// bearing:
///
/// - The `RUNTIME` thread-local borrow is live across `f`. A preemption-
/// driven context switch inside `f` can resume the actor on a DIFFERENT OS
/// thread; the borrow guard would then increment this thread's RefCell but
/// decrement the other's — a count underflow that leaves that thread's
/// RefCell permanently "mutably borrowed". Holding a thread-local guard
/// across a potential switch point is the one unforgivable sin of green
/// threads; disabling preemption makes "no switch inside `f`" structural
/// instead of an accident of which bodies happen to allocate.
/// - `f` is runtime bookkeeping. Suspending an actor halfway through it (or
/// unwinding via the stop sentinel, which shares the gate) is never wanted.
// Borrow the current runtime. Panics if called outside `Runtime::run()`.
//
// Preemption is disabled for the whole span. `f` holds a thread-local borrow
// of `RUNTIME`; if a preemption-driven context switch moved the actor to a
// different OS thread in the middle of `f`, the borrow guard would be
// released on the wrong thread's copy of the thread-local, corrupting its
// borrow count. `f` is also always runtime bookkeeping that should run to
// completion without the actor being suspended or unwound partway through.
pub(crate) fn with_runtime<R>(f: impl FnOnce(&Arc<RuntimeInner>) -> R) -> R {
let prev = crate::preempt::PREEMPTION_ENABLED.with(|c| c.replace(false));
let result = RUNTIME.with(|r| {
@@ -49,9 +101,9 @@ pub(crate) fn with_runtime<R>(f: impl FnOnce(&Arc<RuntimeInner>) -> R) -> R {
result
}
/// Borrow the runtime if present; returns `None` otherwise.
/// Used on cleanup paths (channel Drop during teardown).
/// Same preemption gate as [`with_runtime`]; same reasons.
// Borrow the runtime if present, otherwise `None`. Used on cleanup paths
// (e.g. a channel's Drop impl during teardown) that may run after the
// runtime has already gone away. Same preemption gate as `with_runtime`.
pub(crate) fn try_with_runtime<R>(f: impl FnOnce(&Arc<RuntimeInner>) -> R) -> Option<R> {
let prev = crate::preempt::PREEMPTION_ENABLED.with(|c| c.replace(false));
let result = RUNTIME.with(|r| r.borrow().as_ref().map(f));
@@ -63,19 +115,49 @@ pub(crate) fn try_with_runtime<R>(f: impl FnOnce(&Arc<RuntimeInner>) -> R) -> Op
// JoinHandle / JoinError
// ---------------------------------------------------------------------------
/// The spawned actor panicked. Returned by [`JoinHandle::join`]; `payload`
/// is exactly what the panic carried (the value passed to `panic!`, or
/// whatever a library panicked with), the same payload you'd get from
/// [`std::thread::JoinHandle::join`]. Downcast it if you need to inspect it:
///
/// ```
/// use smarm::{run, spawn};
///
/// run(|| {
/// let h = spawn(|| panic!("boom"));
/// let err = h.join().unwrap_err();
/// let msg = err.payload.downcast_ref::<&str>().copied().unwrap_or("?");
/// assert_eq!(msg, "boom");
/// });
/// ```
#[derive(Debug)]
pub struct JoinError {
pub payload: Box<dyn std::any::Any + Send>,
}
/// A handle to a spawned actor, returned by [`spawn`], [`spawn_under`], and
/// friends. Use [`join`](Self::join) to wait for the actor to finish and
/// collect its outcome, or [`pid`](Self::pid) to get its identity for use
/// with [`request_stop`], [`monitor`](crate::monitor::monitor), or
/// [`link`](crate::link::link).
///
/// If you never call `join` (or drop the handle instead), the actor is not
/// affected: it keeps running and its resources are still reclaimed when it
/// finishes. `join` is how you find out *what happened*, not a requirement
/// for the actor to make progress or clean up.
pub struct JoinHandle {
pid: Pid,
consumed: bool,
}
impl JoinHandle {
/// The identity of the actor this handle refers to.
pub fn pid(&self) -> Pid { self.pid }
/// Block the calling actor until the spawned actor finishes, then
/// report how it finished: `Ok(())` if it returned normally or stopped
/// cooperatively via [`request_stop`], `Err(`[`JoinError`]`)` if it
/// panicked.
pub fn join(mut self) -> Result<(), JoinError> {
use crate::actor::Outcome;
@@ -177,6 +259,41 @@ impl Drop for JoinHandle {
// spawn / spawn_under / self_pid
// ---------------------------------------------------------------------------
/// Per-spawn stack shape overrides (RFC 019). `None` fields resolve to the
/// runtime's [`Config`](crate::runtime::Config) defaults at spawn time, so
/// struct-update syntax works anywhere without a runtime handle:
///
/// ```
/// use smarm::SpawnOpts;
/// let opts = SpawnOpts { stack_reserve: Some(8 * 1024 * 1024), ..SpawnOpts::default() };
/// ```
///
/// Both sizes are page-rounded. The reserve is *virtual* (demand-paged):
/// an 8 MiB reserve costs address space, not memory — RSS follows touched
/// pages. The guard is PROT_NONE below the stack; raise it for FFI code
/// with unusually large C frames. Custom-shaped stacks bypass the recycle
/// pool: they are mmapped fresh at spawn and munmapped at death.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub struct SpawnOpts {
/// Usable stack reservation. `None` ⇒ [`Config::stack_reserve`](crate::runtime::Config::stack_reserve).
pub stack_reserve: Option<usize>,
/// PROT_NONE guard below the stack. `None` ⇒ [`Config::stack_guard`](crate::runtime::Config::stack_guard).
pub guard_size: Option<usize>,
}
/// Start a new actor running `f`, and return a [`JoinHandle`] for it.
///
/// The new actor runs concurrently with its caller and with every other
/// actor in the runtime; smarm schedules it cooperatively across the
/// available OS threads. `spawn` can be called from inside `run`'s closure,
/// or from inside any actor (spawning a child from a child works the same
/// way); it cannot be called before `run` has started or after it returns.
///
/// The returned [`JoinHandle`] is how you learn how the actor finished. If
/// you don't need that, it's fine to drop it: the actor still runs to
/// completion either way.
///
/// Panics if called outside `Runtime::run()`.
pub fn spawn(f: impl FnOnce() + Send + 'static) -> JoinHandle {
let parent = current_pid().unwrap_or_else(|| {
// Outside an actor but inside run(): the initial spawn. with_runtime
@@ -186,17 +303,34 @@ pub fn spawn(f: impl FnOnce() + Send + 'static) -> JoinHandle {
spawn_under(parent, f)
}
/// [`spawn`] with per-actor stack shape overrides (RFC 019).
pub fn spawn_with(opts: SpawnOpts, f: impl FnOnce() + Send + 'static) -> JoinHandle {
let parent = current_pid().unwrap_or_else(|| {
with_runtime(|_| crate::runtime::ROOT_PID)
});
spawn_under_with(parent, opts, f)
}
/// Like [`spawn`], but explicitly attaches the new actor to `supervisor`
/// instead of the calling actor. Ordinary code should reach for [`spawn`];
/// this exists for supervision trees (see [`supervisor`](crate::supervisor))
/// and other cases that need to place a child under a specific ancestor
/// rather than its true caller.
pub fn spawn_under<A>(supervisor: Pid<A>, f: impl FnOnce() + Send + 'static) -> JoinHandle {
spawn_under_with(supervisor, SpawnOpts::default(), f)
}
/// [`spawn_under`] with per-actor stack shape overrides (RFC 019).
pub fn spawn_under_with<A>(
supervisor: Pid<A>,
opts: SpawnOpts,
f: impl FnOnce() + Send + 'static,
) -> JoinHandle {
let supervisor = supervisor.erase();
// Stack + closure boxing happen before ANY runtime lock is taken: no
// syscall and no allocation ever stalls another scheduler thread.
let stack = with_runtime(|inner| inner.stack_pool.lock().pop())
.unwrap_or_else(|| {
match crate::stack::Stack::new(crate::runtime::ACTOR_STACK_SIZE) {
Ok(stack) => stack,
Err(e) => panic!("stack allocation failed: {e}"),
}
});
// Stack + closure boxing happen before the slot locks are taken; the
// pool lock inside acquire_stack is dropped before any mmap, so no
// syscall ever stalls another scheduler thread.
let stack = with_runtime(|inner| crate::runtime::acquire_stack(inner, opts));
let sp = init_actor_stack(stack.top(), crate::actor::trampoline);
let closure: crate::runtime::Closure = Box::new(f);
@@ -208,19 +342,20 @@ pub fn spawn_under<A>(supervisor: Pid<A>, f: impl FnOnce() + Send + 'static) ->
JoinHandle { pid, consumed: false }
}
/// Spawn a typed, single-message actor and hand back its identity-bound
/// [`Pid<A>`] (RFC 014's typed-path producer). The runtime makes the actor's
/// inbox, hands the body its [`Receiver<A::Msg>`], installs the sender, and
/// returns the parent a `Pid<A>`.
/// Spawn an actor that other actors can message directly by its [`Pid<A>`],
/// rather than only by holding on to a channel `Sender` you passed it
/// yourself.
///
/// The inbox is published from the parent side **before** the pid is returned
/// (see [`registry::install_for`](crate::registry::install_for)), so the address
/// is live the instant the caller holds it: an immediate
/// [`send_to`](crate::send_to) always resolves, never racing the body's first
/// instruction. The actor is detached — its lifetime is governed by its own
/// logic (an explicit stop message, or returning), like
/// [`GenServerBuilder::start`](crate::GenServerBuilder::start) — so the backing join
/// handle is dropped. Spawns under the current actor (via [`spawn`]).
/// `body` receives the [`Receiver<A::Msg>`](crate::channel::Receiver) smarm
/// creates for it; `spawn_addr` publishes the matching `Sender` and hands
/// back the actor's typed address. That address is usable the instant you
/// hold it: an immediate [`send_to`](crate::send_to) on the returned pid
/// always finds the inbox, even if `body` hasn't started running yet.
///
/// The spawned actor is detached (there is no [`JoinHandle`] to join): its
/// lifetime is up to its own logic, for example running until it receives a
/// stop message, or until the actor decides to return. This mirrors how
/// [`GenServerBuilder::start`](crate::GenServerBuilder::start) works.
///
/// Panics if called outside `Runtime::run()`.
pub fn spawn_addr<A: crate::pid::Addressable>(
@@ -235,8 +370,25 @@ pub fn spawn_addr<A: crate::pid::Addressable>(
crate::pid::assert_type::<A>(pid)
}
/// [`spawn_addr`] with per-actor stack shape overrides (RFC 019).
pub fn spawn_addr_with<A: crate::pid::Addressable>(
opts: SpawnOpts,
body: impl FnOnce(crate::channel::Receiver<A::Msg>) + Send + 'static,
) -> Pid<A> {
let (tx, rx) = crate::channel::channel::<A::Msg>();
let handle = spawn_with(opts, move || body(rx));
let pid = handle.pid();
crate::registry::install_for::<A::Msg>(pid, tx);
crate::pid::assert_type::<A>(pid)
}
use crate::context::init_actor_stack;
/// The identity of the actor currently running. Use it to hand your own
/// address to another actor (for a reply, a monitor, or a link).
///
/// Panics if called outside an actor (for example, from the closure passed
/// to [`run`] itself, before any [`spawn`]).
pub fn self_pid() -> Pid {
match current_pid() {
Some(pid) => pid,
@@ -248,6 +400,12 @@ pub fn self_pid() -> Pid {
// yield_now / park_current / unpark
// ---------------------------------------------------------------------------
/// Voluntarily give up the CPU so another runnable actor gets a turn, then
/// resume as soon as the scheduler gets back around to you. Use this in a
/// long-running, allocation-free loop that you want to stay cooperative with
/// the rest of the runtime (see also the [`check!`](crate::check) macro,
/// which does the same thing conditionally, only when your timeslice has
/// actually run out).
pub fn yield_now() {
runtime::set_yield_intent(YieldIntent::Yield);
unsafe { crate::context::switch_to_scheduler() };
@@ -255,48 +413,44 @@ pub fn yield_now() {
crate::preempt::check_cancelled();
}
// Suspend the current actor until something wakes it (a message arrives, a
// timer fires, a lock is granted, and so on). This is the low-level parking
// primitive that every blocking smarm operation (channel recv, sleep,
// Mutex::lock, IO waits, JoinHandle::join) is built on; application code
// should reach for one of those rather than calling this directly.
//
// Checks for a pending cooperative-stop request both before parking (a stop
// requested while merely queued to run would otherwise have no future wake
// to catch it) and after resuming (so a stop that arrived while parked is
// noticed as soon as we wake, and unwinds from here exactly like any other
// blocking call would).
pub fn park_current() {
// Entry-side observation point: a stop flagged while we were QUEUED is
// otherwise lost — the stop's wildcard unpark no-ops on a Queued actor
// (the pending run "is" the wake), so if our first action on resume is
// this park, no wake is ever coming and the wake-side check below is
// unreachable. Checking here closes that hole; a flag that lands after
// this check is covered by the existing protocol (the stop's unpark
// finds Running / the prep-to-park window, sets Notified, and the
// park-return re-queues us into the wake-side check). Unwinding from
// here is the same unwind path as the wake side: leftover wait
// registrations are stale-epoch / dead-generation and self-clean at
// their wakers' failed CAS.
crate::preempt::check_cancelled();
runtime::set_yield_intent(YieldIntent::Park);
unsafe { crate::context::switch_to_scheduler() };
// Observation point on the wakeup side of every blocking primitive
// (recv/sleep/mutex/io/join). Past the prep-to-park window, so this never
// races a wakeup: a stop unparks us, we resume here, and unwind out of
// whatever blocking call parked us — running Drop along the way.
crate::preempt::check_cancelled();
}
// Wake `pid` unconditionally, regardless of what it's currently waiting for.
// Reserved for terminal wakes (`request_stop`); anything waking an actor from
// a specific registered wait (a channel send, a mutex grant, a timer, an IO
// completion) must use `unpark_at` instead, so a stale wakeup can never be
// mistaken for the one the actor is actually waiting on.
pub fn unpark(pid: Pid) {
// The whole protocol lives on the slot's packed word (gen + epoch +
// state in one CAS) — see slot_state.rs. No runtime lock unless we
// enqueue. WILDCARD form: reserved for terminal wakes (request_stop);
// every registration-based waker must use `unpark_at`.
let _ = try_with_runtime(|inner| inner.unpark(pid));
}
/// Epoch-matched unpark: wake `pid` only if its current wait is still the
/// one this waker registered for. The form every registration-based waker
/// (channel senders, mutex grants, wait-timers, io completions, joiner
/// wakes) must use — see slot_state.rs for the consuming-wake rules.
// Wake `pid` only if its current wait is still the one this waker
// registered for (an "epoch-matched" unpark). Every registration-based
// waker (channel senders, mutex grants, wait-timers, IO completions, joiner
// wakes) must use this rather than the unconditional `unpark`.
pub(crate) fn unpark_at(pid: Pid, epoch: u32) {
let _ = try_with_runtime(|inner| inner.unpark_at(pid, epoch));
}
/// Open a new wait for the CURRENT actor: bump its park-epoch and return
/// it. Call once per wait, before registering `(pid, epoch)` with any
/// waker. Lock-free (one CAS on the own slot word), so it is legal under
/// any lock, including a Channel-class lock.
// Open a new wait for the current actor and return its wait identity
// ("epoch"). Call once per wait, before registering with any waker. Lock-free,
// so it's legal to call while already holding another internal lock.
pub(crate) fn begin_wait() -> u32 {
let me = match current_pid() {
Some(pid) => pid,
@@ -305,48 +459,47 @@ pub(crate) fn begin_wait() -> u32 {
with_runtime(|inner| inner.begin_wait(me))
}
/// Close the current actor's wait WITHOUT parking on it. For the no-park
/// exit of multi-registration waits (`select` finding an arm ready at
/// registration time): bumps the epoch so in-flight wakes die at their CAS,
/// eats a notification that already landed, then observes a pending stop —
/// in that order. After this, leftover registrations are stale-epoch and
/// self-clean at their wakers' failed CAS; nothing can fault the actor's
/// next one-shot park.
// Close the current actor's wait without parking on it: the no-park exit
// used when a multi-arm `select` finds an arm already ready at registration
// time. Leftover registrations from the other arms are left to self-clean
// when their wakers try to use them.
pub(crate) fn retire_wait() {
let me = match current_pid() {
Some(pid) => pid,
None => panic!("retire_wait() called outside an actor"),
};
with_runtime(|inner| inner.retire_wait(me));
// A request_stop that fired before the clear had its notification
// eaten, but it set the stop flag first — observe it here and unwind.
// One that fires after re-notifies a Running word as usual.
crate::preempt::check_cancelled();
}
/// Request cooperative cancellation of `pid`.
/// Ask `pid` to stop cooperatively.
///
/// Sets the actor's stop flag and wakes it so it observes the flag promptly:
/// a parked target is re-queued; a running target is marked notified, so its
/// next park returns immediately to an observation point. The actor realises
/// the stop as a controlled unwind at its next observation point
/// (`check!()`/allocation, or the wakeup side of a blocking park),
/// terminating with `Outcome::Stopped`.
/// This sets a flag on the target actor and wakes it so it notices promptly;
/// the actor itself decides when it's safe to actually unwind, at its next
/// natural checkpoint (a blocking call returning, a `check!()`, or an
/// allocation). Once it does, it terminates as if it panicked, except that
/// [`JoinHandle::join`] reports it as a normal, non-error exit: cooperative
/// stop is a controlled shutdown, not a failure.
///
/// This is best-effort and cooperative: an actor that never reaches an
/// observation point — a tight loop with no `check!()`, no allocation, and no
/// blocking op — cannot be stopped, exactly as it cannot be preempted. A no-op
/// if `pid` is already gone.
/// This is exactly the mechanism `gen_server` shutdown, supervisor restarts,
/// and structured teardown are built from: reach for [`GenServerRef::shutdown`](crate::GenServerRef::shutdown)
/// or a [`supervisor`](crate::supervisor) instead of calling this directly
/// where those apply.
///
/// Because it's cooperative, an actor stuck in a tight loop with no
/// blocking call, no [`check!`](crate::check), and no allocation cannot be
/// stopped, for the same reason it cannot be preempted. Calling this on an
/// actor that has already finished is a harmless no-op.
pub fn request_stop<A>(pid: Pid<A>) {
let pid = pid.erase();
let _ = try_with_runtime(|inner| request_stop_inner(inner, pid));
}
/// The core of [`request_stop`], taking the runtime directly so it can be
/// driven from inside the runtime (e.g. the root-exit sweep in
/// `finalize_actor`) without re-borrowing the thread-local. Sets the stop flag
/// under the target's cold lock (generation re-verified there; a mismatch or a
/// slot with no live actor is a no-op) and wakes it.
// The core of `request_stop`, taking the runtime directly so it can also be
// driven from inside the runtime itself (the root-exit sweep) without
// re-borrowing the thread-local. Sets the stop flag under the target's lock
// (a generation mismatch, or no live actor there, makes it a no-op) and
// wakes the target.
pub(crate) fn request_stop_inner(inner: &RuntimeInner, pid: Pid) {
if let Some(slot) = inner.slot_at(pid) {
{
@@ -367,6 +520,10 @@ pub(crate) fn request_stop_inner(inner: &RuntimeInner, pid: Pid) {
// NoPreempt
// ---------------------------------------------------------------------------
/// A guard that disables preemption for its lifetime, restoring the
/// previous setting on drop. Internal-use: application code has no need to
/// disable preemption directly. See [`check!`](crate::check) for the
/// user-facing side of preemption.
pub struct NoPreempt(bool);
impl NoPreempt {
@@ -386,6 +543,21 @@ impl Drop for NoPreempt {
// sleep / insert_wait_timer
// ---------------------------------------------------------------------------
/// Suspend the calling actor for `duration`. Unlike
/// [`std::thread::sleep`], this parks only the actor, not the underlying OS
/// thread, so every other actor (including others sharing the same OS
/// thread) keeps running normally while this one waits.
///
/// ```
/// use smarm::{run, sleep};
/// use std::time::Duration;
///
/// run(|| {
/// sleep(Duration::from_millis(1));
/// });
/// ```
///
/// Panics if called outside an actor.
pub fn sleep(duration: std::time::Duration) {
let me = match current_pid() {
Some(pid) => pid,
@@ -403,11 +575,11 @@ pub fn sleep(duration: std::time::Duration) {
park_current();
}
/// Like [`sleep`], but wall-anchored: under causal profiling (feature
/// `smarm-causal`) the deadline is honoured in wall time instead of chasing
/// injected virtual delay. Identical to [`sleep`] without the feature. For
/// measurement machinery whose durations define wall time (the causal
/// controller's windows, TSC calibration) — workload code wants [`sleep`].
/// Like [`sleep`], but the deadline is always measured in real (wall-clock)
/// time. Ordinary code should use [`sleep`]; this variant exists for smarm's
/// own profiling and measurement tooling, which can otherwise stretch or
/// compress simulated time. Without that tooling active, `sleep_wall` and
/// `sleep` behave identically.
pub fn sleep_wall(duration: std::time::Duration) {
let me = match current_pid() {
Some(pid) => pid,
@@ -425,6 +597,11 @@ pub fn sleep_wall(duration: std::time::Duration) {
park_current();
}
// Building block for bounded waits elsewhere in the crate (Mutex::lock_timeout,
// Receiver::recv_timeout, select_timeout): arm a timer that, on expiry, asks
// `target` whether this particular wait is still pending and should be woken
// with a timeout. Not part of the public API; application code wants
// `sleep`, `send_after`, or one of the `*_timeout` methods instead.
pub fn insert_wait_timer(
deadline: std::time::Instant,
pid: Pid,
@@ -444,20 +621,22 @@ pub fn insert_wait_timer(
}
// ---------------------------------------------------------------------------
// send_after / cancel_timer — message-delivery timers (Erlang send_after).
//
// Arm a timer that delivers `msg` to an address after `after`, returning a
// `TimerId`. The destination is resolved *on fire*, not at arm time: a
// `Pid<A>` that has since died yields `SendError::Dead`, a `Name<M>` resolves
// to whoever currently holds it (so a restarted server is reached). Either way
// a failed resolve / closed inbox is dropped, matching `erlang:send_after`.
// `cancel_timer` prevents an as-yet-unfired delivery; it returns whether the
// timer was still armed.
// send_after / cancel_timer: message-delivery timers, in the same spirit as
// Erlang's `erlang:send_after/3`. Each schedules `msg` for delivery after a
// delay and returns a TimerId; `cancel_timer` can call one off before it
// fires. The destination is resolved when the timer actually fires, not when
// it's armed, so a `Name`-addressed timer always reaches whoever currently
// holds that name, even if the original holder has since restarted. If the
// destination is gone by fire time, the message is silently dropped, exactly
// as a live send to a dead address would be.
// ---------------------------------------------------------------------------
/// Deliver `msg` to the exact actor named by `dest` (identity-bound, no
/// redirect — see [`send_to`](crate::registry::send_to)) after `after`.
/// Returns a [`TimerId`](crate::timer::TimerId) for [`cancel_timer`].
/// Deliver `msg` to the exact actor identified by `dest` after `after` has
/// elapsed. Unlike [`send_after_named`], this targets one specific actor: if
/// that actor is gone by the time the timer fires, the message is dropped
/// (it is never redirected to a different actor, even one that inherited the
/// same name). Returns a [`TimerId`](crate::timer::TimerId) you can pass to
/// [`cancel_timer`] to call it off early.
pub fn send_after<A: crate::pid::Addressable>(
after: std::time::Duration,
dest: Pid<A>,
@@ -475,9 +654,11 @@ pub fn send_after<A: crate::pid::Addressable>(
})
}
/// Deliver `msg` to whichever actor holds the name `dest` at fire time
/// (re-resolving [`send`](crate::registry::send) semantics) after `after`.
/// Returns a [`TimerId`](crate::timer::TimerId) for [`cancel_timer`].
/// Deliver `msg` to whichever actor holds the name `dest` when the timer
/// fires, not necessarily whoever holds it now: if the named actor restarts
/// (for example under a supervisor) and re-registers before the deadline,
/// the message reaches the new instance. Returns a
/// [`TimerId`](crate::timer::TimerId) you can pass to [`cancel_timer`].
pub fn send_after_named<M: Send + 'static>(
after: std::time::Duration,
dest: Name<M>,
@@ -497,13 +678,12 @@ pub fn send_after_named<M: Send + 'static>(
})
}
/// Wall-anchored [`send_after`] (RFC 007): under causal profiling the timer
/// opts out of the virtual-time shift and fires at its raw deadline instead
/// of dilating with injected delay — the user-facing opt-out whose substrate
/// [`sleep_wall`] landed. Use it for deadlines that reflect the outside
/// world (protocol timeouts, wall-clock schedules) rather than workload
/// pacing. Without the `smarm-causal` feature it is identical to
/// [`send_after`]. Cancellation via [`cancel_timer`] is unchanged.
/// Like [`send_after`], but the deadline is always measured in real
/// (wall-clock) time rather than being subject to smarm's own profiling and
/// measurement tooling. Use this for deadlines that need to reflect the
/// outside world (a protocol timeout, a wall-clock schedule) rather than
/// simulated workload pacing. Without that tooling active, it behaves
/// identically to [`send_after`].
pub fn send_after_wall<A: crate::pid::Addressable>(
after: std::time::Duration,
dest: Pid<A>,
@@ -521,8 +701,8 @@ pub fn send_after_wall<A: crate::pid::Addressable>(
})
}
/// Wall-anchored [`send_after_named`] (RFC 007): re-resolving name delivery,
/// raw-deadline anchor — see [`send_after_wall`] for the semantics.
/// Wall-clock-anchored [`send_after_named`]: re-resolving name delivery,
/// with the same real-time deadline guarantee as [`send_after_wall`].
pub fn send_after_named_wall<M: Send + 'static>(
after: std::time::Duration,
dest: Name<M>,
@@ -542,17 +722,12 @@ pub fn send_after_named_wall<M: Send + 'static>(
})
}
/// Deliver `msg` onto a channel the caller owns after `after`, rather than to a
/// registry address. The sibling of [`send_after`] used by the gen_server timer
/// layer (RFC 015 §5): arming a server timer must land the fire on the loop's
/// own `Sys` channel — giving it the loop's arm position — not in the inbox.
///
/// Same substrate as [`send_after`]: a `Reason::Send` entry, the same `armed`
/// set, the same [`TimerId`](crate::timer::TimerId) for [`cancel_timer`]. Only
/// the fire thunk differs — `tx.send(msg)` instead of a registry resolve — so a
/// send onto a channel whose receiver is gone is dropped, exactly as a failed
/// address resolve is (Erlang `send_after` semantics). `pid` is informational
/// (who armed it); delivery lives entirely in the thunk.
// Deliver `msg` onto a channel the caller already holds, after a delay,
// rather than resolving a registry address at fire time. Used internally by
// gen_server's timer support, which needs the fire to land on the server
// loop's own dedicated channel instead of its public inbox. Otherwise
// identical to `send_after`: same cancellation via `cancel_timer`, and a
// send to a channel whose receiver is gone is silently dropped.
pub(crate) fn send_after_to<T: Send + 'static>(
after: std::time::Duration,
tx: Sender<T>,
@@ -571,9 +746,9 @@ pub(crate) fn send_after_to<T: Send + 'static>(
})
}
/// Cancel a timer armed by [`send_after`] / [`send_after_named`]. Returns
/// `true` if it was still pending (delivery now prevented), `false` if it had
/// already fired or been cancelled.
/// Call off a timer armed by [`send_after`] or [`send_after_named`] before
/// it fires. Returns `true` if the timer was still pending and delivery is
/// now prevented, `false` if it had already fired or was already cancelled.
pub fn cancel_timer(id: crate::timer::TimerId) -> bool {
with_runtime(|inner| {
match inner.timers.lock() {
@@ -587,6 +762,22 @@ pub fn cancel_timer(id: crate::timer::TimerId) -> bool {
// block_on_io / wait_readable / wait_writable / read / write
// ---------------------------------------------------------------------------
/// Run a blocking closure (blocking file IO, a synchronous library call,
/// anything that isn't itself actor-aware) on a dedicated worker thread,
/// while the calling actor parks and every other actor keeps making
/// progress. When `f` completes, the calling actor resumes with its result.
///
/// Reach for this whenever you need to call into something that would
/// otherwise block the underlying OS thread outright: a blocking C library,
/// a synchronous filesystem call, DNS resolution via the system resolver,
/// and so on. For plain readiness-based network IO on a file descriptor,
/// [`wait_readable`] / [`wait_writable`] are cheaper since they don't need a
/// dedicated thread.
///
/// If `f` panics, that panic is carried over and re-raised in the calling
/// actor, exactly as if the call had been made inline.
///
/// Panics if called outside an actor.
pub fn block_on_io<F, T>(f: F) -> T
where
F: FnOnce() -> T + Send + 'static,
@@ -609,7 +800,14 @@ where
Err(e) => panic!("smarm: io lock poisoned (core corrupt): {e}"),
};
match io.as_mut() {
Some(io) => io.submit(me, epoch, work),
Some(io) => {
// RFC 018: count the op in flight BEFORE submit — the
// pool decrements on completion, and an increment that
// trailed the completion would underflow. Under the io
// lock, so ordered against the same-lock submit.
inner.io_outstanding.fetch_add(1, Ordering::AcqRel);
io.submit(me, epoch, work);
}
None => panic!("io thread not started"),
}
});
@@ -639,10 +837,19 @@ where
}
}
/// Park the calling actor until `fd` becomes readable. Other actors keep
/// running while you wait; when the kernel reports the fd ready, the actor
/// resumes and you perform the actual `read(2)` yourself (see [`read`] for
/// a convenience wrapper that does both steps).
///
/// Only one actor may wait on a given fd for a given direction at a time.
/// Panics if called outside an actor.
pub fn wait_readable(fd: std::os::fd::RawFd) -> std::io::Result<()> {
wait_fd(fd, true, false)
}
/// Park the calling actor until `fd` becomes writable. See [`wait_readable`]
/// for the read-side counterpart; the same notes apply.
pub fn wait_writable(fd: std::os::fd::RawFd) -> std::io::Result<()> {
wait_fd(fd, false, true)
}
@@ -660,18 +867,28 @@ fn wait_fd(fd: std::os::fd::RawFd, readable: bool, writable: bool) -> std::io::R
Err(e) => panic!("smarm: io lock poisoned (core corrupt): {e}"),
};
match io.as_mut() {
Some(io) => io.epoll_register(fd, me, epoch, readable, writable),
Some(io) => {
// RFC 018: count the waiter BEFORE the ADD (mirror of
// submit); roll back if the registration fails so a
// rejected wait leaves the verdict counters clean.
inner.io_fd_waiters.fetch_add(1, Ordering::AcqRel);
let r = io.epoll_register(fd, me, epoch, readable, writable);
if r.is_err() {
inner.io_fd_waiters.fetch_sub(1, Ordering::AcqRel);
}
r
}
None => panic!("io thread not started"),
}
})?;
// If a terminal stop unwinds us out of the park below, the registration
// must not outlive us: a stale `waiters` entry fails every future
// `wait_*` on this fd with AlreadyExists, and the kernel-side ADD leaks
// until the fd happens to be reused. Clean up iff the entry is still
// ours — a `FdReady` racing the stop may have consumed it already (it
// removes + DELs under the io lock), after which the fd may even carry
// ANOTHER actor's fresh registration; in that case touch nothing.
// must not outlive us: a stale registration would fail every future
// `wait_*` on this fd, and the kernel-side registration would leak until
// the fd happens to be reused. Clean up only if the entry is still
// ours; a wakeup racing the stop may have already consumed it, possibly
// leaving a different actor's fresh registration in its place, which
// must not be disturbed.
struct Dereg {
fd: std::os::fd::RawFd,
me: Pid,
@@ -685,9 +902,12 @@ fn wait_fd(fd: std::os::fd::RawFd, readable: bool, writable: bool) -> std::io::R
Err(e) => panic!("smarm: io lock poisoned (core corrupt): {e}"),
};
if let Some(io) = io.as_mut() {
if io.waiters.get(&self.fd) == Some(&(self.me, self.epoch)) {
io.waiters.remove(&self.fd);
io.epoll_deregister(self.fd);
// `cancel_waiter` removes + DELs iff still ours, all
// under the waiters lock (the ADD/DEL serialization);
// decrement only when we actually removed it — a
// FdReady that consumed it already did the decrement.
if io.cancel_waiter(self.fd, self.me, self.epoch) {
inner.io_fd_waiters.fetch_sub(1, Ordering::AcqRel);
}
}
});
@@ -695,24 +915,28 @@ fn wait_fd(fd: std::os::fd::RawFd, readable: bool, writable: bool) -> std::io::R
}
let guard = Dereg { fd, me, epoch };
park_current();
// Normal wake: the FdReady path removed the entry and DEL'd the fd
// before the unpark, so the guard's check would be a guaranteed no-op —
// skip the io lock on the hot path. (Dereg owns nothing; forget leaks
// no resource.)
// Normal wake: the ready-fd path already removed the entry and
// deregistered from epoll before waking us, so the guard's check on drop
// is a guaranteed no-op. Skip it on this hot path (Dereg owns no
// resource itself, so forgetting it leaks nothing).
std::mem::forget(guard);
Ok(())
}
// ---------------------------------------------------------------------------
// FdArm — fd readiness as a select arm (RFC 008)
// FdArm: fd readiness as a select arm
// ---------------------------------------------------------------------------
/// An fd-readiness arm for [`crate::select`] / [`crate::select_timeout`]:
/// ready when the fd is readable (resp. writable), composable with channel
/// receivers on one wait epoch. Phase-1 rules apply: one waiter per fd at a
/// time, one direction per arm (duplex on a single fd needs `dup`; epoll
/// registrations key on the open file description, so dup'd fds register
/// independently).
/// A file-descriptor readiness condition usable as an arm of
/// [`select`](crate::select) / [`select_timeout`](crate::select_timeout),
/// so you can wait on "this fd is readable" alongside ordinary channel
/// receivers in the same call. Build one with [`FdArm::readable`] or
/// [`FdArm::writable`].
///
/// Only one actor may wait on a given fd for a given direction at a time. A
/// single fd open in both directions needs two `FdArm`s (or `dup` the fd) if
/// you want to wait on both; you cannot wait on read and write readiness
/// with one arm.
pub struct FdArm {
fd: std::os::fd::RawFd,
readable: bool,
@@ -720,10 +944,12 @@ pub struct FdArm {
}
impl FdArm {
/// An arm that becomes ready when `fd` is readable.
pub fn readable(fd: std::os::fd::RawFd) -> Self {
FdArm { fd, readable: true, writable: false }
}
/// An arm that becomes ready when `fd` is writable.
pub fn writable(fd: std::os::fd::RawFd) -> Self {
FdArm { fd, readable: false, writable: true }
}
@@ -732,14 +958,12 @@ impl FdArm {
impl crate::channel::sealed::Sealed for FdArm {}
impl crate::channel::Selectable for FdArm {
/// Ready-now check is a zero-timeout `poll(2)`; if the requested events
/// are pending the wait is retired without registering (`Ok(false)`,
/// the channel-arm contract). Otherwise register with the io thread —
/// every failure surfaces as `Err` (EBADF including a closed-fd
/// POLLNVAL, EMFILE on the epoll set, AlreadyExists for a second
/// waiter on the fd): the fallible-out, nothing-left-behind rule, in
/// deviation from RFC 008's permanently-ready lean, which would spin a
/// consumer whose fd is healthy but unregistrable (EMFILE).
// Ready-now check is a zero-timeout poll(2); if the requested events are
// already pending, the wait is retired without registering. Otherwise
// register with the IO thread. Any registration failure (a closed fd,
// too many fds registered, a second waiter already on this fd) is
// surfaced as an error rather than silently treated as "always ready",
// so a caller never spins on an fd that genuinely can't be registered.
fn sel_register(&self, pid: Pid, epoch: u32) -> std::io::Result<bool> {
if poll_events(self.fd, self.readable, self.writable)? {
return Ok(false);
@@ -751,7 +975,14 @@ impl crate::channel::Selectable for FdArm {
};
match io.as_mut() {
Some(io) => {
io.epoll_register(self.fd, pid, epoch, self.readable, self.writable)
inner.io_fd_waiters.fetch_add(1, Ordering::AcqRel);
let r = io.epoll_register(
self.fd, pid, epoch, self.readable, self.writable,
);
if r.is_err() {
inner.io_fd_waiters.fetch_sub(1, Ordering::AcqRel);
}
r
}
None => panic!("io thread not started"),
}
@@ -759,20 +990,19 @@ impl crate::channel::Selectable for FdArm {
Ok(true)
}
/// Classification is the same zero-timeout poll: a pure function of fd
/// state, independent of the registration the cleanup pass removed. An
/// error here (EBADF: fd closed mid-wait) reports READY — the
/// consumer's read/write surfaces the errno; a dead arm is an event,
/// not a hang.
// Same zero-timeout poll used for classification: a pure function of
// fd state. An error here (fd closed mid-wait) is reported as ready
// rather than pending forever: the caller's own read/write then
// surfaces the real error, so a dead fd is an observable event, not a
// silent hang.
fn sel_ready(&self) -> bool {
poll_events(self.fd, self.readable, self.writable).unwrap_or(true)
}
/// `wait_fd`'s `Dereg` compare, verbatim: remove the waiters entry and
/// kernel-side registration iff the entry is still `(pid, epoch)`-ours.
/// A `FdReady` racing the wake may have consumed it (it removes + DELs
/// under the io lock), after which the fd may even carry ANOTHER
/// actor's fresh registration; in that case touch nothing.
// Mirrors wait_fd's cleanup guard: remove the registration only if it's
// still ours. A wakeup racing this cleanup may have already consumed
// it, possibly leaving a different actor's fresh registration on the
// same fd, which must not be touched.
fn sel_unregister(&self, pid: Pid, epoch: u32) {
with_runtime(|inner| {
let mut io = match inner.io.lock() {
@@ -780,9 +1010,8 @@ impl crate::channel::Selectable for FdArm {
Err(e) => panic!("smarm: io lock poisoned (core corrupt): {e}"),
};
if let Some(io) = io.as_mut() {
if io.waiters.get(&self.fd) == Some(&(pid, epoch)) {
io.waiters.remove(&self.fd);
io.epoll_deregister(self.fd);
if io.cancel_waiter(self.fd, pid, epoch) {
inner.io_fd_waiters.fetch_sub(1, Ordering::AcqRel);
}
}
});
@@ -793,9 +1022,9 @@ impl crate::channel::Selectable for FdArm {
}
}
/// Zero-timeout `poll(2)`: are any of the requested events (or ERR/HUP,
/// which make the consumer's read/write fail loudly rather than park
/// forever) pending on `fd`? POLLNVAL maps to `Err(EBADF)`.
// Zero-timeout poll(2): are any of the requested events (or an error/hangup
// condition, so the caller's read/write fails loudly instead of parking
// forever) pending on `fd` right now?
fn poll_events(fd: std::os::fd::RawFd, readable: bool, writable: bool) -> std::io::Result<bool> {
let mut events: libc::c_short = 0;
if readable {
@@ -824,8 +1053,9 @@ fn poll_events(fd: std::os::fd::RawFd, readable: bool, writable: bool) -> std::i
}
}
/// Wait until `fd` is readable or `timeout` elapses: `Ok(true)` = ready,
/// `Ok(false)` = timed out. A one-arm [`crate::try_select_timeout`].
/// Like [`wait_readable`], but gives up after `timeout` instead of waiting
/// indefinitely: `Ok(true)` means the fd became ready, `Ok(false)` means the
/// timeout elapsed first.
pub fn wait_readable_timeout(
fd: std::os::fd::RawFd,
timeout: std::time::Duration,
@@ -834,8 +1064,9 @@ pub fn wait_readable_timeout(
Ok(crate::channel::try_select_timeout(&[&arm], timeout)?.is_some())
}
/// Wait until `fd` is writable or `timeout` elapses: `Ok(true)` = ready,
/// `Ok(false)` = timed out.
/// Like [`wait_writable`], but gives up after `timeout` instead of waiting
/// indefinitely: `Ok(true)` means the fd became ready, `Ok(false)` means the
/// timeout elapsed first.
pub fn wait_writable_timeout(
fd: std::os::fd::RawFd,
timeout: std::time::Duration,
@@ -844,12 +1075,18 @@ pub fn wait_writable_timeout(
Ok(crate::channel::try_select_timeout(&[&arm], timeout)?.is_some())
}
/// Convenience wrapper: park until `fd` is readable, then perform the
/// `read(2)` into `buf`. Equivalent to calling [`wait_readable`] yourself
/// followed by a raw read, provided as a shorthand for the common case.
pub fn read(fd: std::os::fd::RawFd, buf: &mut [u8]) -> std::io::Result<usize> {
wait_readable(fd)?;
let n = unsafe { libc::read(fd, buf.as_mut_ptr() as *mut _, buf.len()) };
if n < 0 { Err(std::io::Error::last_os_error()) } else { Ok(n as usize) }
}
/// Convenience wrapper: park until `fd` is writable, then perform the
/// `write(2)` of `buf`. Equivalent to calling [`wait_writable`] yourself
/// followed by a raw write, provided as a shorthand for the common case.
pub fn write(fd: std::os::fd::RawFd, buf: &[u8]) -> std::io::Result<usize> {
wait_writable(fd)?;
let n = unsafe { libc::write(fd, buf.as_ptr() as *const _, buf.len()) };
@@ -857,7 +1094,7 @@ pub fn write(fd: std::os::fd::RawFd, buf: &[u8]) -> std::io::Result<usize> {
}
// ---------------------------------------------------------------------------
// register_supervisor_channel
// register_supervisor_channel: internal wiring used by supervisor.rs
// ---------------------------------------------------------------------------
pub fn register_supervisor_channel(pid: Pid, sender: Sender<Signal>) {
@@ -874,11 +1111,21 @@ pub fn register_supervisor_channel(pid: Pid, sender: Sender<Signal>) {
}
// ---------------------------------------------------------------------------
// Legacy run() — convenience wrapper
// run(): the single-threaded convenience entry point
// ---------------------------------------------------------------------------
/// Single-threaded runtime entry point (backwards-compatible wrapper).
/// Equivalent to `runtime::init(Config::exact(1)).run(f)`.
/// Start the smarm runtime on a single OS thread, run `f` as the first
/// (root) actor, and block until every actor it transitively spawned has
/// finished.
///
/// This is the simplest way to get started, and is all you need for most
/// programs: `f` typically calls [`spawn`] to create more actors and waits
/// on their [`JoinHandle`]s. If you want smarm to schedule actors across
/// multiple OS threads instead, use
/// [`crate::runtime::init`] with a
/// [`Config`](crate::runtime::Config) that requests more than one scheduler
/// thread, then call [`Runtime::run`](crate::runtime::Runtime::run) on it;
/// `run` here is exactly that, pinned to one thread.
pub fn run<F: FnOnce() + Send + 'static>(f: F) {
crate::runtime::init(crate::runtime::Config::exact(1)).run(f);
}
+321
View File
@@ -0,0 +1,321 @@
//! RFC 019 §7 — overflow diagnostics.
//!
//! One process-global SIGSEGV handler, installed once at [`crate::runtime::init`]
//! (before any scheduler thread exists, so the PRIOR save is unracing), plus a
//! per-scheduler-thread `sigaltstack` registered at `schedule_loop` entry — a
//! guard hit means the faulting stack has no room to run anything, so the
//! altstack is not optional.
//!
//! The handler classifies `si_addr` against the *current* actor only, reached
//! through `preempt::CURRENT_SLOT` — a const-initialized `Cell<*const Slot>`
//! whose access is a plain TLS load (no lazy init, no allocation, no dtor
//! registration), and which every scheduler thread has materialized before an
//! actor can run on it. The slot's diag atomics (`diag_stack_top` & co) are
//! written in `install_actor` before the Release publish and are only consulted
//! here while the actor is on-CPU, so they cannot be stale.
//!
//! Two classification tiers:
//! - **In-guard**: definitive. Rust frames probe pages in order
//! (`__rust_probestack`), so Rust overflow always lands here; so does any C
//! built with `-fstack-clash-protection` (distro-packaged libraries), and —
//! with the 1 MiB default guard — nearly every unprobed frame too.
//! - **Overshoot**: within [`OVERSHOOT_SLOP`] *below* the guard. An unprobed
//! frame (cargo-built C via `cc` almost never enables clash protection)
//! large enough to step over the guard in one `sub rsp`. Attribution is
//! "probable": the address is in unmapped VA that nothing else owns, an
//! actor was on-CPU, and the distance fits a frame — the diagnostic says so.
//!
//! Classified faults print one line (async-signal-safe: stack buffer +
//! `write(2)`, no fmt, no alloc, no locks) and re-raise with default
//! disposition — no unwind, no resume, no fail-soft (jarred; UB-adjacent from
//! a handler). Unclassified faults reinstate the PRIOR handler and refault, so
//! std's own "thread ... has overflowed its stack" diagnostics for OS-thread
//! stacks survive our presence. Reinstating deregisters us for good, which is
//! fine: the process is dying either way.
use std::cell::Cell;
use std::mem::MaybeUninit;
use std::sync::atomic::Ordering;
use std::sync::Once;
/// Tier-2 window below the guard. Matches the guard default (and the kernel's
/// `stack_guard_gap`): a frame that out-jumps both the guard and this window
/// in one displacement is past what a diagnostic can honestly attribute.
pub(crate) const OVERSHOOT_SLOP: usize = 1024 * 1024;
/// Per-scheduler-thread signal stack. MINSIGSTKSZ is ~11 KiB on AVX-512
/// hardware; 64 KiB leaves the formatter room without mattering to anyone.
/// One per OS thread, never freed: scheduler threads live for the process in
/// practice, and repeated `run()`s on reused threads re-use the registration
/// (the TLS flag), so the leak is bounded by the OS thread count.
const ALTSTACK_SIZE: usize = 64 * 1024;
static INSTALL: Once = Once::new();
/// The handler that was installed before ours (std's, typically). Written
/// exactly once inside INSTALL — which completes in `runtime::init` before
/// any scheduler thread (and thus any classifiable fault) can exist — and
/// only read from the handler afterwards.
static mut PRIOR: MaybeUninit<libc::sigaction> = MaybeUninit::uninit();
thread_local! {
/// Whether this OS thread has registered its altstack.
static ALTSTACK_SET: Cell<bool> = const { Cell::new(false) };
}
/// Where a fault landed relative to the current actor's stack.
#[derive(Debug, PartialEq, Eq)]
pub(crate) enum FaultClass {
/// Inside `[top − reserve − guard, top − reserve)`: the guard region.
Guard,
/// Within `OVERSHOOT_SLOP` below the guard: stepped over it. Payload is
/// the distance below `guard_lo`.
Overshoot(usize),
/// Not ours to explain.
Foreign,
}
/// Pure classifier — all edges unit-tested below. `top` is the stack's usable
/// top, `reserve`/`guard` its shape; both page-rounded by `Stack::new`.
pub(crate) fn classify(addr: usize, top: usize, reserve: usize, guard: usize) -> FaultClass {
let guard_hi = top.wrapping_sub(reserve);
let guard_lo = guard_hi.wrapping_sub(guard);
if addr >= guard_lo && addr < guard_hi {
FaultClass::Guard
} else if addr < guard_lo && addr >= guard_lo.saturating_sub(OVERSHOOT_SLOP) {
FaultClass::Overshoot(guard_lo - addr)
} else {
FaultClass::Foreign
}
}
/// Install the process-global handler. Idempotent; called from
/// `runtime::init`.
pub(crate) fn install_once() {
INSTALL.call_once(|| unsafe {
let mut sa: libc::sigaction = std::mem::zeroed();
sa.sa_sigaction = handler as *const () as usize;
sa.sa_flags = libc::SA_SIGINFO | libc::SA_ONSTACK;
libc::sigemptyset(&mut sa.sa_mask);
let prior = &mut *std::ptr::addr_of_mut!(PRIOR);
libc::sigaction(libc::SIGSEGV, &sa, prior.as_mut_ptr());
});
}
/// Register this OS thread's altstack (idempotent per thread). Called at
/// `schedule_loop` entry, so every thread that can run an actor has one.
pub(crate) fn register_altstack() {
ALTSTACK_SET.with(|set| {
if set.get() {
return;
}
unsafe {
let sp = libc::mmap(
std::ptr::null_mut(),
ALTSTACK_SIZE,
libc::PROT_READ | libc::PROT_WRITE,
libc::MAP_PRIVATE | libc::MAP_ANONYMOUS,
-1,
0,
);
if sp == libc::MAP_FAILED {
// Degrade: no altstack means a guard hit dies without the
// message (handler can't run) — the pre-RFC behavior, never
// incorrectness.
return;
}
let ss = libc::stack_t {
ss_sp: sp,
ss_flags: 0,
ss_size: ALTSTACK_SIZE,
};
libc::sigaltstack(&ss, std::ptr::null_mut());
}
set.set(true);
});
}
// ---------------------------------------------------------------------------
// The handler
// ---------------------------------------------------------------------------
unsafe extern "C" fn handler(
_sig: libc::c_int,
info: *mut libc::siginfo_t,
_ctx: *mut libc::c_void,
) {
let slot_ptr = crate::preempt::current_slot_ptr();
if !slot_ptr.is_null() {
let slot = &*slot_ptr;
let top = slot.diag_stack_top.load(Ordering::Relaxed);
if top != 0 {
let reserve = slot.diag_stack_reserve.load(Ordering::Relaxed);
let guard = slot.diag_stack_guard.load(Ordering::Relaxed);
let pid = slot.diag_pid.load(Ordering::Relaxed);
let addr = (*info).si_addr() as usize;
match classify(addr, top, reserve, guard) {
FaultClass::Guard => {
let mut b = Buf::new();
b.s("smarm: actor ");
b.pid(pid);
b.s(" overflowed its stack: fault in the guard region, depth-at-fault=");
b.u(top - addr);
b.s(" bytes (reserve=");
b.u(reserve);
b.s(", guard=");
b.u(guard);
b.s("). Raise stack_reserve (SpawnOpts or Config).\n");
b.emit();
die_by_default();
return;
}
FaultClass::Overshoot(below) => {
let mut b = Buf::new();
b.s("smarm: actor ");
b.pid(pid);
b.s(" probably overflowed its stack: fault ");
b.u(below);
b.s(" bytes below the guard - an unprobed (FFI?) frame stepped over it (reserve=");
b.u(reserve);
b.s(", guard=");
b.u(guard);
b.s("). Raise stack_guard or stack_reserve.\n");
b.emit();
die_by_default();
return;
}
FaultClass::Foreign => {}
}
}
}
// Not ours: put back whoever was there before us and refault into them.
let prior = &*std::ptr::addr_of!(PRIOR);
libc::sigaction(libc::SIGSEGV, prior.as_ptr(), std::ptr::null_mut());
}
/// Reset SIGSEGV to default disposition; returning from the handler then
/// refaults at the same instruction and the process dies the normal death
/// (core-dumpable, correct wait status), exactly as if we were never here —
/// but with the message already on stderr.
unsafe fn die_by_default() {
let mut dfl: libc::sigaction = std::mem::zeroed();
dfl.sa_sigaction = libc::SIG_DFL;
libc::sigemptyset(&mut dfl.sa_mask);
libc::sigaction(libc::SIGSEGV, &dfl, std::ptr::null_mut());
}
// ---------------------------------------------------------------------------
// Async-signal-safe formatting: fixed buffer, decimal itoa, one write(2).
// ---------------------------------------------------------------------------
struct Buf {
b: [u8; 320],
len: usize,
}
impl Buf {
fn new() -> Self {
Buf { b: [0; 320], len: 0 }
}
fn s(&mut self, s: &str) {
for &c in s.as_bytes() {
if self.len < self.b.len() {
self.b[self.len] = c;
self.len += 1;
}
}
}
fn u(&mut self, mut n: usize) {
let mut tmp = [0u8; 20];
let mut i = tmp.len();
loop {
i -= 1;
tmp[i] = b'0' + (n % 10) as u8;
n /= 10;
if n == 0 {
break;
}
}
for &c in &tmp[i..] {
if self.len < self.b.len() {
self.b[self.len] = c;
self.len += 1;
}
}
}
/// `idx.gen`, unpacked from the install-time packing.
fn pid(&mut self, packed: u64) {
self.u((packed >> 32) as usize);
self.s(".");
self.u((packed & 0xffff_ffff) as usize);
}
fn emit(&self) {
unsafe {
libc::write(2, self.b.as_ptr() as *const libc::c_void, self.len);
}
}
}
// ---------------------------------------------------------------------------
// Classifier units — the arithmetic edges, before anything integrates.
// ---------------------------------------------------------------------------
#[cfg(test)]
mod tests {
use super::{classify, FaultClass, OVERSHOOT_SLOP};
const PG: usize = 4096;
// A synthetic stack far from address-space edges: top at 1 GiB.
const TOP: usize = 1 << 30;
const RESERVE: usize = 16 * PG;
const GUARD: usize = 4 * PG;
const GUARD_HI: usize = TOP - RESERVE;
const GUARD_LO: usize = GUARD_HI - GUARD;
#[test]
fn inside_guard_both_edges() {
assert_eq!(classify(GUARD_LO, TOP, RESERVE, GUARD), FaultClass::Guard);
assert_eq!(classify(GUARD_HI - 1, TOP, RESERVE, GUARD), FaultClass::Guard);
assert_eq!(classify(GUARD_LO + GUARD / 2, TOP, RESERVE, GUARD), FaultClass::Guard);
}
#[test]
fn usable_region_is_foreign() {
// A fault inside the RW stack itself isn't a guard hit and must not
// be explained as one.
assert_eq!(classify(GUARD_HI, TOP, RESERVE, GUARD), FaultClass::Foreign);
assert_eq!(classify(TOP - 1, TOP, RESERVE, GUARD), FaultClass::Foreign);
}
#[test]
fn above_top_is_foreign() {
assert_eq!(classify(TOP, TOP, RESERVE, GUARD), FaultClass::Foreign);
assert_eq!(classify(TOP + PG, TOP, RESERVE, GUARD), FaultClass::Foreign);
}
#[test]
fn overshoot_window_edges() {
assert_eq!(
classify(GUARD_LO - 1, TOP, RESERVE, GUARD),
FaultClass::Overshoot(1)
);
assert_eq!(
classify(GUARD_LO - OVERSHOOT_SLOP, TOP, RESERVE, GUARD),
FaultClass::Overshoot(OVERSHOOT_SLOP)
);
assert_eq!(
classify(GUARD_LO - OVERSHOOT_SLOP - 1, TOP, RESERVE, GUARD),
FaultClass::Foreign
);
}
#[test]
fn low_address_stack_saturates_not_wraps() {
// A stack mapped so low that the slop window would underflow: the
// window clips to 0 instead of wrapping around the address space.
let top = RESERVE + GUARD + PG; // guard_lo == PG
assert_eq!(classify(0, top, RESERVE, GUARD), FaultClass::Overshoot(PG));
// Null-page fault still classified only because it IS within slop
// here; with a normal-height stack it is Foreign (covered above by
// the window-edge test at realistic addresses).
}
}
+227 -14
View File
@@ -1,32 +1,45 @@
//! mmap-based growable stack with a guard page below.
//! mmap-based actor stack with a PROT_NONE guard region below (RFC 019).
//!
//! Layout (low → high address):
//! [ guard page (PROT_NONE) | stack region ]
//! ^ top() — initial stack pointer
//! [ guard region (PROT_NONE) | stack region ]
//! ^ top() — initial stack pointer
//!
//! Stacks grow downward. Overflow lands in the guard page → SIGSEGV.
//! Stacks grow downward. Overflow lands in the guard region → SIGSEGV.
//!
//! Both the usable reserve and the guard are caller-chosen (page-rounded).
//! The reserve is a *virtual* reservation: anonymous mmap is demand-paged,
//! so RSS is touched-pages, not reserve × actors. The guard costs address
//! space only. A wide guard (the runtime defaults to 64 KiB) exists for
//! unprobed FFI frames: Rust frames touch pages in order (probestack), so
//! one page catches Rust overflow, but a C frame with a large local can
//! step over a single page in one `sub rsp`.
use std::io;
pub struct Stack {
/// Bottom of the entire mmap'd region (start of guard page).
/// Bottom of the entire mmap'd region (start of the guard).
base: *mut u8,
/// Total mmap'd size: guard_size + stack_size.
total_size: usize,
/// Usable stack size (excluding guard page).
/// Usable stack size (excluding the guard).
stack_size: usize,
/// PROT_NONE region below the usable stack.
guard_size: usize,
}
// Stack owns its memory; safe to send across threads.
unsafe impl Send for Stack {}
impl Stack {
/// Allocate a new stack. `stack_size` is the usable region; one page is
/// added below as a guard page. Both are rounded up to the page size.
pub fn new(stack_size: usize) -> io::Result<Self> {
/// Allocate a new stack. `stack_size` is the usable region; `guard_size`
/// is mapped PROT_NONE below it. Both are rounded up to the page size
/// and must be non-zero.
pub fn new(stack_size: usize, guard_size: usize) -> io::Result<Self> {
assert!(stack_size > 0, "stack_size must be non-zero");
assert!(guard_size > 0, "guard_size must be non-zero");
let page = page_size();
let stack_size = round_up(stack_size, page);
let guard_size = page;
let guard_size = round_up(guard_size, page);
let total_size = guard_size + stack_size;
let base = unsafe {
@@ -53,7 +66,7 @@ impl Stack {
return Err(err);
}
Ok(Self { base, total_size, stack_size })
Ok(Self { base, total_size, stack_size, guard_size })
}
/// 16-byte-aligned top of the usable region.
@@ -62,14 +75,54 @@ impl Stack {
(raw_top & !15) as *mut u8
}
/// Pointer to the bottom of the usable region (just above the guard page).
/// Pointer to the bottom of the usable region (just above the guard).
pub fn usable_base(&self) -> *mut u8 {
unsafe { self.base.add(page_size()) }
unsafe { self.base.add(self.guard_size) }
}
pub fn stack_size(&self) -> usize {
self.stack_size
}
pub fn guard_size(&self) -> usize {
self.guard_size
}
/// `(stack_size, guard_size)` after page rounding. The pool rule
/// (RFC 019 §1) compares this against the runtime defaults: only
/// default-shaped stacks are pooled.
pub fn shape(&self) -> (usize, usize) {
(self.stack_size, self.guard_size)
}
/// Pool-recycle zap (RFC 019 §6): `MADV_DONTNEED` everything below the
/// retained entry end `[top − retain, top)` — the span the next actor's
/// shallow frames land in stays resident, the dead spike below it is
/// released. The stack is unowned at the call site (its actor is dead),
/// so a synchronous eager zap races nothing and the RSS drop is
/// immediate — a museum of worst-case spikes is exactly what a pool must
/// not be; DONTNEED's ~8× per-page cost vs FREE is irrelevant off the
/// hot path. Advisory like the park-path shrink: a failure degrades to
/// "the pool keeps RSS", never to incorrectness. No-op (no syscall) when
/// `retain` covers the whole usable region — i.e. always, at the 64 KiB
/// default reserve.
pub(crate) fn recycle_zap(&self, retain: usize) {
if let Some((off, len)) = retain_range(self.stack_size, retain, page_size()) {
unsafe {
libc::madvise(
self.usable_base().add(off) as *mut libc::c_void,
len,
libc::MADV_DONTNEED,
);
}
}
}
}
/// Round `n` up to whole pages — the same rounding `Stack::new` applies, so
/// runtime defaults stored pre-rounded compare exactly against [`Stack::shape`].
pub(crate) fn round_to_pages(n: usize) -> usize {
round_up(n, page_size())
}
impl Drop for Stack {
@@ -80,10 +133,170 @@ impl Drop for Stack {
}
}
fn page_size() -> usize {
pub(crate) fn page_size() -> usize {
unsafe { libc::sysconf(libc::_SC_PAGESIZE) as usize }
}
fn round_up(n: usize, align: usize) -> usize {
(n + align - 1) & !(align - 1)
}
/// The whole-page span the park-path shrink may `MADV_FREE` (RFC 019 §3):
/// `[page_up(hwm), page_down(sp − redzone))`, or `None` if no full page fits.
///
/// `hwm` is the sampled high-water (deepest observed `sp`); everything in
/// `[hwm, sp)` is below the live frame and dead by definition. One page of
/// redzone stays resident under live `sp` — it covers the SysV 128-byte red
/// zone plus spill margin with room to spare. Rounding is inward on both
/// ends so the result can never touch the redzone, cross `sp`, or dip below
/// `hwm`; all arithmetic is checked so adversarial inputs (`sp < redzone`,
/// `hwm ≥ sp`, values near the address-space edges) collapse to `None`
/// rather than a wild or negative-length range.
pub(crate) fn shrink_range(hwm: usize, sp: usize, page: usize) -> Option<(usize, usize)> {
debug_assert!(page.is_power_of_two());
if hwm >= sp {
return None;
}
let redzone = page;
let end = sp.checked_sub(redzone)? & !(page - 1); // page_down(sp − redzone)
let start = hwm.checked_add(page - 1)? & !(page - 1); // page_up(hwm)
if end > start {
Some((start, end - start))
} else {
None
}
}
/// The `(offset_from_usable_base, len)` span the pool recycle DONTNEEDs
/// (RFC 019 §6): everything below the retained entry end. "Bottom RETAIN of
/// the stack" is read stack-wise (entry frames = highest addresses of a
/// downward stack): the retained span is `[top − page_up(retain), top)`, the
/// zapped span is the rest — retaining the low-address deep end instead
/// would keep the coldest pages and release the ones the next actor faults
/// first. `retain` rounds *up* to whole pages (retain more, zap less), so
/// with `stack_size` page-rounded by `Stack::new` the result is always
/// page-aligned. Checked math: `retain ≥ stack_size` (notably the default
/// 64 KiB reserve with the 64 KiB RETAIN) and overflow collapse to `None`.
pub(crate) fn retain_range(stack_size: usize, retain: usize, page: usize) -> Option<(usize, usize)> {
debug_assert!(page.is_power_of_two());
let retain = retain.checked_add(page - 1)? & !(page - 1); // page_up(retain)
let len = stack_size.checked_sub(retain)?;
if len == 0 {
return None;
}
Some((0, len))
}
#[cfg(test)]
mod tests {
use super::{retain_range, shrink_range};
const PG: usize = 4096;
#[test]
fn retain_covers_whole_stack_is_a_noop() {
// The default config: reserve == RETAIN == 64 KiB. No zap, no syscall.
assert_eq!(retain_range(16 * PG, 16 * PG, PG), None);
assert_eq!(retain_range(PG, PG, PG), None);
}
#[test]
fn retain_larger_than_stack_is_a_noop() {
assert_eq!(retain_range(16 * PG, 17 * PG, PG), None);
assert_eq!(retain_range(PG, usize::MAX, PG), None); // page_up overflows
}
#[test]
fn retain_zero_zaps_everything() {
assert_eq!(retain_range(16 * PG, 0, PG), Some((0, 16 * PG)));
}
#[test]
fn retain_rounds_up_zapping_less() {
// 1 byte of retain keeps a whole page.
assert_eq!(retain_range(16 * PG, 1, PG), Some((0, 15 * PG)));
assert_eq!(retain_range(16 * PG, PG + 1, PG), Some((0, 14 * PG)));
}
#[test]
fn retain_one_page_short_of_stack() {
assert_eq!(retain_range(2 * PG, PG, PG), Some((0, PG)));
}
#[test]
fn retain_range_is_page_aligned() {
for size_pg in [1usize, 2, 3, 16, 1024] {
for retain in [0usize, 1, PG - 1, PG, PG + 1, 4 * PG, size_pg * PG] {
if let Some((off, len)) = retain_range(size_pg * PG, retain, PG) {
assert_eq!(off, 0);
assert_eq!(len % PG, 0);
assert!(len <= size_pg * PG);
assert!(len > 0);
}
}
}
}
#[test]
fn empty_and_inverted_spans_are_none() {
assert_eq!(shrink_range(0x8000_0000, 0x8000_0000, PG), None); // hwm == sp
assert_eq!(shrink_range(0x8000_1000, 0x8000_0000, PG), None); // hwm > sp
}
#[test]
fn span_smaller_than_redzone_plus_page_is_none() {
let sp = 0x8000_0000;
// Everything within redzone+1 page of sp: no full page clears both
// the redzone and the page_up(hwm) rounding.
assert_eq!(shrink_range(sp - PG, sp, PG), None);
assert_eq!(shrink_range(sp - 2 * PG + 1, sp, PG), None);
}
#[test]
fn exact_two_pages_frees_one() {
let sp = 0x8000_0000;
let hwm = sp - 2 * PG;
// [hwm, hwm+PG) frees; [sp−PG, sp) is redzone.
assert_eq!(shrink_range(hwm, sp, PG), Some((hwm, PG)));
}
#[test]
fn unaligned_ends_round_inward() {
let sp = 0x8000_0123; // live sp mid-page
let hwm = 0x7f00_0abc; // high-water mid-page
let (start, len) = shrink_range(hwm, sp, PG).unwrap();
assert_eq!(start % PG, 0);
assert_eq!(len % PG, 0);
assert!(start >= hwm); // never below the sampled high-water
assert!(start + len <= (sp - PG) & !(PG - 1)); // never into the redzone
}
#[test]
fn result_never_crosses_sp() {
// Sweep hwm across every offset of the page straddling the boundary.
let sp = 0x8000_0000 + 137;
for hwm in (sp - 4 * PG)..(sp) {
if let Some((start, len)) = shrink_range(hwm, sp, PG) {
assert!(start >= hwm);
assert!(start + len + PG <= sp + PG); // end ≤ page_down(sp − PG) < sp
assert!(len > 0);
}
}
}
#[test]
fn underflow_near_zero_is_none() {
assert_eq!(shrink_range(0, PG - 1, PG), None); // sp < redzone
assert_eq!(shrink_range(0, 0, PG), None);
}
#[test]
fn big_span_frees_interior() {
let sp = 0x8000_0000;
let spike = 4 * 1024 * 1024;
let hwm = sp - spike;
let (start, len) = shrink_range(hwm, sp, PG).unwrap();
assert_eq!(start, hwm); // aligned input: starts exactly at hwm
assert_eq!(len, spike - PG); // everything but the redzone page
}
}
+11 -2
View File
@@ -6,10 +6,19 @@
//! Build the loom models with: `RUSTFLAGS="--cfg loom" cargo test --lib --release`
#[cfg(loom)]
pub(crate) use loom::sync::atomic::{AtomicU64, AtomicUsize, Ordering};
pub(crate) use loom::sync::atomic::{fence, AtomicU64, AtomicUsize, Ordering};
#[cfg(not(loom))]
pub(crate) use std::sync::atomic::{AtomicU64, AtomicUsize, Ordering};
pub(crate) use std::sync::atomic::{fence, AtomicU64, AtomicUsize, Ordering};
// park.rs condvar-parker (loom + non-Linux builds only; the Linux non-loom
// build parks on a futex and never touches these — gating them identically
// keeps the default build free of unused imports).
#[cfg(loom)]
pub(crate) use loom::sync::{Condvar, Mutex};
#[cfg(all(not(loom), not(target_os = "linux")))]
pub(crate) use std::sync::{Condvar, Mutex};
/// `UnsafeCell` with loom's `with`/`with_mut` access API; pass-through cost
/// is zero in normal builds (`#[inline]`, newtype over std's cell).
+37 -1
View File
@@ -141,6 +141,14 @@ impl PartialOrd for Entry {
#[derive(Default)]
pub struct Timers {
/// RFC 018: the scheduler coordination layer. Attached once at
/// `RuntimeInner::new`; every insert notes its deadline (min-maintained
/// snapshot for the busy-path due-check + the timekeeper re-arm wake)
/// and every pop/clear re-anchors the snapshot to the heap minimum.
/// All calls happen under the timers mutex — the serialization the
/// coordinator's timer protocol mandates. `None` only in unit tests
/// that construct a bare `Timers`.
coord: Option<std::sync::Arc<crate::park::Coordinator>>,
/// Reverse-wrapped so the smallest deadline is at the top.
heap: BinaryHeap<Reverse<Entry>>,
/// Monotonic counter for the tiebreaker `seq` field (and the `TimerId` of a
@@ -157,7 +165,18 @@ pub struct Timers {
impl Timers {
pub fn new() -> Self {
Self { heap: BinaryHeap::new(), next_seq: 0, armed: std::collections::HashSet::new() }
Self {
coord: None,
heap: BinaryHeap::new(),
next_seq: 0,
armed: std::collections::HashSet::new(),
}
}
/// Attach the scheduler coordination layer (RFC 018). Called once, at
/// runtime construction, before any scheduler thread exists.
pub(crate) fn attach_coordinator(&mut self, c: std::sync::Arc<crate::park::Coordinator>) {
self.coord = Some(c);
}
/// Insert a `Sleep` timer. Convenience for the common case.
@@ -242,6 +261,13 @@ impl Timers {
#[cfg(feature = "smarm-causal")]
wall,
}));
// RFC 018: publish the (possibly new-minimum) deadline to the
// busy-path snapshot and wake the timekeeper if it is parked
// toward a later one. We hold the timers mutex — the mandated
// serialization for both.
if let Some(c) = &self.coord {
c.note_deadline(deadline);
}
seq
}
@@ -255,6 +281,9 @@ impl Timers {
pub fn clear(&mut self) {
self.heap.clear();
self.armed.clear();
if let Some(c) = &self.coord {
c.refresh_deadline(None);
}
}
/// Soonest pending deadline, or `None` if the heap is empty.
@@ -324,6 +353,13 @@ impl Timers {
}
out.push(entry);
}
// RFC 018: re-anchor the busy-path snapshot to the new heap minimum
// (still under the timers mutex). A causal-shift re-queue above went
// through `heap.push` directly, so this peek is the one place the
// snapshot is guaranteed to catch up.
if let Some(c) = &self.coord {
c.refresh_deadline(self.peek_deadline());
}
out
}
}
+5 -5
View File
@@ -23,7 +23,7 @@ extern "C-unwind" fn actor_simple() {
#[test]
fn actor_runs_and_returns_to_scheduler() {
reset_log();
let stack = Stack::new(64 * 1024).unwrap();
let stack = Stack::new(64 * 1024, 4096).unwrap();
let sp = init_actor_stack(stack.top(), actor_simple);
set_actor_sp(sp);
unsafe { switch_to_actor() };
@@ -40,7 +40,7 @@ extern "C-unwind" fn actor_two_steps() {
#[test]
fn actor_yields_and_resumes() {
reset_log();
let stack = Stack::new(64 * 1024).unwrap();
let stack = Stack::new(64 * 1024, 4096).unwrap();
let sp = init_actor_stack(stack.top(), actor_two_steps);
set_actor_sp(sp);
@@ -85,7 +85,7 @@ extern "C-unwind" fn actor_reg_check() {
#[test]
fn callee_saved_registers_survive_yield() {
let stack = Stack::new(64 * 1024).unwrap();
let stack = Stack::new(64 * 1024, 4096).unwrap();
let sp = init_actor_stack(stack.top(), actor_reg_check);
set_actor_sp(sp);
unsafe { switch_to_actor(); switch_to_actor(); }
@@ -117,8 +117,8 @@ extern "C-unwind" fn actor_b() {
#[test]
fn two_actors_dont_corrupt_each_other() {
let stack_a = Stack::new(64 * 1024).unwrap();
let stack_b = Stack::new(64 * 1024).unwrap();
let stack_a = Stack::new(64 * 1024, 4096).unwrap();
let stack_b = Stack::new(64 * 1024, 4096).unwrap();
let sp_a = init_actor_stack(stack_a.top(), actor_a);
let sp_b = init_actor_stack(stack_b.top(), actor_b);
+126
View File
@@ -237,6 +237,13 @@ fn tree_from_nests_children_and_reroots_orphans() {
overruns: 0,
messages_received: 0,
budget_cycles: 0,
stack: smarm::StackInfo {
reserve: 0,
guard: 0,
depth_high_water: 0,
parks_since_shrink: 0,
shrinks: 0,
},
};
let snap = RuntimeSnapshot {
@@ -352,3 +359,122 @@ fn budget_cycles_accumulate_when_enabled() {
h.join().unwrap();
});
}
// ---------------------------------------------------------------------------
// RFC 019 §8 — the stack introspection surface.
// ---------------------------------------------------------------------------
/// Burn ~`frames` × 4 KiB of stack with a yield at max depth, so the context
/// save samples the high-water there (RFC 019 §2: hwm is SAMPLED at
/// deschedule, not tracked continuously).
#[inline(never)]
fn burn_stack_yielding(frames: usize) -> u64 {
let mut local = [0u8; 4096];
local[0] = frames as u8;
let below = if frames == 0 {
smarm::yield_now();
0
} else {
burn_stack_yielding(frames - 1)
};
std::hint::black_box(&mut local);
below.wrapping_add(local[0] as u64)
}
#[test]
fn stack_info_reports_defaults_and_sampled_depth() {
run(|| {
let (ready_tx, ready_rx) = channel::<()>();
let (gate_tx, gate_rx) = channel::<()>();
let h = spawn(move || {
// ~32 KiB deep with a yield at the bottom: the sample point.
std::hint::black_box(burn_stack_yielding(8));
ready_tx.send(()).unwrap();
gate_rx.recv().unwrap();
});
ready_rx.recv().unwrap();
let info = spin_until(h.pid(), |a| a.state == ActorState::Parked);
let s = info.stack;
assert_eq!(s.reserve, 64 * 1024, "default reserve");
assert_eq!(s.guard, 1024 * 1024, "default guard (kernel stack_guard_gap convention)");
assert!(
s.depth_high_water >= 8 * 4096,
"hwm sampled at the deep yield: expected ≥ 32 KiB, got {}",
s.depth_high_water
);
assert!(
s.depth_high_water < s.reserve,
"depth {} cannot exceed the reserve {}",
s.depth_high_water,
s.reserve
);
// Parked at the gate right now, never shrunk (64 KiB reserve cannot
// cross the shrink threshold).
assert!(s.parks_since_shrink >= 1, "the gate park must be counted");
assert_eq!(s.shrinks, 0);
gate_tx.send(()).unwrap();
h.join().unwrap();
});
}
#[test]
fn stack_info_shrink_counters_are_live() {
use smarm::runtime::{Config, SHRINK_COOLDOWN, SHRINK_THRESHOLD};
use smarm::{spawn_with, SpawnOpts};
let rt = smarm::runtime::init(Config::exact(1));
rt.run(|| {
let (park_tx, park_rx) = channel::<()>();
let spike = 768 * 4096;
assert!(spike > SHRINK_THRESHOLD);
let worker = spawn_with(
SpawnOpts { stack_reserve: Some(8 * 1024 * 1024), ..SpawnOpts::default() },
move || {
std::hint::black_box(burn_stack_yielding(768));
for _ in 0..(SHRINK_COOLDOWN + 8) {
park_rx.recv().unwrap();
}
},
);
let wpid = worker.pid();
// Before any parks complete: the spike depth is visible.
let info = spin_until(wpid, |a| a.state == ActorState::Parked);
assert!(
info.stack.depth_high_water >= spike,
"spike should be sampled: {} < {spike}",
info.stack.depth_high_water
);
// Cross the cooldown, then read the counters live while the worker
// is parked waiting for the remaining rounds (post-join the slot is
// reclaimed and the generation check correctly hides it).
for _ in 0..(SHRINK_COOLDOWN + 2) {
spin_until(wpid, |a| a.state == ActorState::Parked);
park_tx.send(()).unwrap();
}
let info = spin_until(wpid, |a| a.state == ActorState::Parked && a.stack.shrinks >= 1);
let s = info.stack;
assert!(s.shrinks >= 1, "cooldown was crossed with a spike above threshold");
assert!(
s.parks_since_shrink < SHRINK_COOLDOWN,
"counter must reset at shrink: {}",
s.parks_since_shrink
);
assert!(
s.depth_high_water < spike,
"hwm resets to the shallow park sp at shrink; got {}",
s.depth_high_water
);
for _ in 0..6 {
spin_until(wpid, |a| a.state == ActorState::Parked);
park_tx.send(()).unwrap();
}
worker.join().unwrap();
});
}
+69
View File
@@ -0,0 +1,69 @@
//! RFC 018 scheduler park/wake — observable-behavior guards.
//!
//! These pin the two timer-latency properties the park/wake swap must
//! preserve or introduce:
//!
//! - `sleep_fires_under_saturation`: due timers fire even when every
//! scheduler is busy (nobody parked ⇒ no timekeeper) — the busy-path
//! due-check, ratified design point (a). The old drain phase gave this
//! for free (timers drained every loop iteration); the new design must
//! not lose it.
//! - `submillisecond_sleep_is_prompt`: a sub-ms sleep completes promptly.
//! Under the old wake pipe, `poll_wake`'s `as_millis` truncation turned
//! sub-ms deadlines into 0ms busy-polls (correct wall time, pathological
//! CPU); under park/wake the futex timespec carries full nanosecond
//! precision.
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use std::time::{Duration, Instant};
#[test]
fn sleep_fires_under_saturation() {
let rt = smarm::runtime::init(smarm::runtime::Config::exact(4));
rt.run(|| {
let stop = Arc::new(AtomicBool::new(false));
let mut spinners = Vec::new();
// 8 spinners over 4 schedulers: the run queue never empties, so no
// scheduler ever parks and no timekeeper exists. Only the busy-path
// due-check can fire the sleeper's timer before the spinners quit.
for _ in 0..8 {
let stop = stop.clone();
spinners.push(smarm::spawn(move || {
let t0 = Instant::now();
while !stop.load(Ordering::Relaxed) && t0.elapsed() < Duration::from_secs(5) {
smarm::yield_now();
}
}));
}
let t0 = Instant::now();
smarm::sleep(Duration::from_millis(10));
let dt = t0.elapsed();
stop.store(true, Ordering::Relaxed);
for s in spinners {
let _ = s.join();
}
assert!(
dt < Duration::from_millis(500),
"10ms sleep took {dt:?} under scheduler saturation — busy-path \
timer firing is broken (timekeeper-only firing stalls under load)"
);
});
}
#[test]
fn submillisecond_sleep_is_prompt() {
let rt = smarm::runtime::init(smarm::runtime::Config::exact(2));
rt.run(|| {
// Warm one iteration, then measure.
smarm::sleep(Duration::from_micros(500));
let t0 = Instant::now();
smarm::sleep(Duration::from_micros(500));
let dt = t0.elapsed();
assert!(dt >= Duration::from_micros(400), "woke early: {dt:?}");
assert!(
dt < Duration::from_millis(100),
"500µs sleep took {dt:?} — sub-ms deadline handling is broken"
);
});
}
+34
View File
@@ -517,3 +517,37 @@ fn runtime_reusable_after_root_panic() {
r.run(move || ran_t.store(true, Ordering::Relaxed));
assert!(ran.load(Ordering::Relaxed), "runtime unusable after root panic");
}
// ---------------------------------------------------------------------------
// RFC 019 — Config stack knobs
// ---------------------------------------------------------------------------
/// Burn ~`frames` × 4 KiB of stack; probestack touches pages in order so
/// exceeding the reserve would hit the guard and SIGSEGV the process.
#[inline(never)]
fn burn_stack(frames: usize) -> u64 {
let mut local = [0u8; 4096];
local[0] = frames as u8;
let below = if frames == 0 { 0 } else { burn_stack(frames - 1) };
std::hint::black_box(&mut local);
below.wrapping_add(local[0] as u64)
}
#[test]
fn config_stack_reserve_permits_deep_recursion() {
// ~256 KiB of frames: four times the old fixed 64 KiB reserve. With
// Config::stack_reserve raised this must complete; before RFC 019 it
// could only segfault.
let rt = smarm::runtime::init(Config::exact(1).stack_reserve(1024 * 1024));
let done = Arc::new(AtomicBool::new(false));
let done2 = done.clone();
rt.run(move || {
spawn(move || {
std::hint::black_box(burn_stack(64));
done2.store(true, Ordering::SeqCst);
})
.join()
.unwrap();
});
assert!(done.load(Ordering::SeqCst));
}
+212
View File
@@ -0,0 +1,212 @@
//! RFC 019 commit 2 — the `SpawnOpts` surface.
//!
//! Covers: per-spawn stack shape overrides on every spawn surface, the
//! `None ⇒ Config default` resolution, the pool rule from the outside
//! (obligation 4: a custom-shaped stack never enters the pool), and that a
//! big reserve behaviorally takes effect (deep recursion completes).
use smarm::runtime::{Config, DEFAULT_STACK_GUARD, DEFAULT_STACK_RESERVE};
use smarm::{
self_pid, spawn, spawn_under_with, spawn_with, GenServerBuilder, SpawnOpts,
};
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
fn rt1() -> smarm::runtime::Runtime {
smarm::runtime::init(Config::exact(1))
}
#[test]
fn default_spawn_has_default_shape() {
rt1().run(|| {
let h = spawn(|| {
let shape = smarm::introspect::stack_shape(self_pid()).unwrap();
assert_eq!(shape, (DEFAULT_STACK_RESERVE, DEFAULT_STACK_GUARD));
});
h.join().unwrap();
});
}
#[test]
fn spawn_with_overrides_reserve_and_guard() {
rt1().run(|| {
let opts = SpawnOpts {
stack_reserve: Some(1024 * 1024),
guard_size: Some(256 * 1024),
};
let h = spawn_with(opts, || {
let shape = smarm::introspect::stack_shape(self_pid()).unwrap();
assert_eq!(shape, (1024 * 1024, 256 * 1024));
});
h.join().unwrap();
});
}
#[test]
fn spawn_with_partial_override_keeps_config_default_for_the_rest() {
rt1().run(|| {
let opts = SpawnOpts { stack_reserve: Some(1024 * 1024), ..SpawnOpts::default() };
let h = spawn_with(opts, || {
let shape = smarm::introspect::stack_shape(self_pid()).unwrap();
assert_eq!(shape, (1024 * 1024, DEFAULT_STACK_GUARD));
});
h.join().unwrap();
});
}
#[test]
fn spawn_with_rounds_to_pages() {
rt1().run(|| {
let opts = SpawnOpts { stack_reserve: Some(64 * 1024 + 1), guard_size: Some(4097) };
let h = spawn_with(opts, || {
let (reserve, guard) = smarm::introspect::stack_shape(self_pid()).unwrap();
assert_eq!(reserve % 4096, 0);
assert_eq!(guard % 4096, 0);
assert!(reserve >= 64 * 1024 + 1);
assert!(guard >= 4097);
});
h.join().unwrap();
});
}
#[test]
fn spawn_under_with_takes_opts() {
rt1().run(|| {
let me = self_pid();
let opts = SpawnOpts { stack_reserve: Some(128 * 1024), ..SpawnOpts::default() };
let h = spawn_under_with(me, opts, || {
let (reserve, _) = smarm::introspect::stack_shape(self_pid()).unwrap();
assert_eq!(reserve, 128 * 1024);
});
h.join().unwrap();
});
}
/// Obligation 4, from the outside: a dead custom stack must not be handed to
/// the next default spawn. The pool is LIFO, so if the custom stack had been
/// (wrongly) pushed at death, the very next default-shaped spawn on this
/// single-threaded runtime would pop it and report a custom shape.
#[test]
fn custom_stack_never_enters_the_pool() {
rt1().run(|| {
spawn_with(
SpawnOpts { stack_reserve: Some(512 * 1024), guard_size: Some(128 * 1024) },
|| {},
)
.join()
.unwrap();
let h = spawn(|| {
let shape = smarm::introspect::stack_shape(self_pid()).unwrap();
assert_eq!(shape, (DEFAULT_STACK_RESERVE, DEFAULT_STACK_GUARD));
});
h.join().unwrap();
});
}
/// The reverse direction of the pool rule: a default-shaped stack IS pooled
/// and reused (cap = threads × 4 ≥ 1 here, pool empty at start).
#[test]
fn default_stack_is_recycled() {
rt1().run(|| {
spawn(|| {}).join().unwrap();
let h = spawn(|| {
let shape = smarm::introspect::stack_shape(self_pid()).unwrap();
assert_eq!(shape, (DEFAULT_STACK_RESERVE, DEFAULT_STACK_GUARD));
});
h.join().unwrap();
});
}
/// Burn ~`frames` × 4 KiB of stack (see tests/runtime.rs twin).
#[inline(never)]
fn burn_stack(frames: usize) -> u64 {
let mut local = [0u8; 4096];
local[0] = frames as u8;
let below = if frames == 0 { 0 } else { burn_stack(frames - 1) };
std::hint::black_box(&mut local);
below.wrapping_add(local[0] as u64)
}
#[test]
fn big_reserve_behaviorally_takes_effect() {
// ~1 MiB deep on an 8 MiB per-spawn reserve, runtime default untouched.
rt1().run(|| {
let done = Arc::new(AtomicBool::new(false));
let done2 = done.clone();
spawn_with(
SpawnOpts { stack_reserve: Some(8 * 1024 * 1024), ..SpawnOpts::default() },
move || {
std::hint::black_box(burn_stack(256));
done2.store(true, Ordering::SeqCst);
},
)
.join()
.unwrap();
assert!(done.load(Ordering::SeqCst));
});
}
// ---------------------------------------------------------------------------
// Builder surfaces
// ---------------------------------------------------------------------------
struct Echo;
impl smarm::GenServer for Echo {
type Call = ();
type Reply = (usize, usize);
type Cast = ();
type Info = ();
type Timer = ();
fn handle_call(&mut self, _c: ()) -> (usize, usize) {
smarm::introspect::stack_shape(self_pid()).unwrap()
}
fn handle_cast(&mut self, _c: ()) {}
}
#[test]
fn gen_server_builder_stack_opts() {
rt1().run(|| {
let server = GenServerBuilder::new(Echo)
.stack_opts(SpawnOpts { stack_reserve: Some(256 * 1024), ..SpawnOpts::default() })
.start();
let (reserve, guard) = server.call(()).unwrap();
assert_eq!(reserve, 256 * 1024);
assert_eq!(guard, DEFAULT_STACK_GUARD);
server.shutdown();
});
}
struct Probe;
impl smarm::Machine for Probe {
type Ev = smarm::channel::Sender<(usize, usize)>;
fn state_timeout_ev() -> Self::Ev {
unreachable!("no timers in this test")
}
fn timeout_ev(_name: &'static str) -> Self::Ev {
unreachable!("no timers in this test")
}
fn on_start(&mut self, _cx: &mut smarm::Cx<Self::Ev>) {}
fn handle(
&mut self,
ev: Self::Ev,
_cx: &mut smarm::Cx<Self::Ev>,
) -> smarm::gen_statem::Step<Self::Ev> {
let _ = ev.send(smarm::introspect::stack_shape(self_pid()).unwrap());
smarm::gen_statem::Step::Stayed
}
}
#[test]
fn gen_statem_spawn_with_stack_opts() {
rt1().run(|| {
let m = smarm::gen_statem::spawn_with(
SpawnOpts { stack_reserve: Some(256 * 1024), ..SpawnOpts::default() },
Probe,
);
let (tx, rx) = smarm::channel::channel();
m.send(tx).unwrap();
let (reserve, guard) = rx.recv().unwrap();
assert_eq!(reserve, 256 * 1024);
assert_eq!(guard, DEFAULT_STACK_GUARD);
});
}
+77 -9
View File
@@ -7,13 +7,13 @@ use smarm::stack::Stack;
#[test]
fn top_is_16_byte_aligned() {
let s = Stack::new(64 * 1024).unwrap();
let s = Stack::new(64 * 1024, 4096).unwrap();
assert_eq!(s.top() as usize % 16, 0);
}
#[test]
fn top_is_within_allocation() {
let s = Stack::new(64 * 1024).unwrap();
let s = Stack::new(64 * 1024, 4096).unwrap();
let top = s.top() as usize;
let base = s.usable_base() as usize;
assert!(top > base);
@@ -22,7 +22,7 @@ fn top_is_within_allocation() {
#[test]
fn write_and_read_top_of_stack() {
let s = Stack::new(64 * 1024).unwrap();
let s = Stack::new(64 * 1024, 4096).unwrap();
let sentinel: u64 = 0xDEAD_BEEF_CAFE_1234;
unsafe {
let ptr = s.top().sub(8) as *mut u64;
@@ -33,7 +33,7 @@ fn write_and_read_top_of_stack() {
#[test]
fn write_and_read_bottom_of_usable_region() {
let s = Stack::new(64 * 1024).unwrap();
let s = Stack::new(64 * 1024, 4096).unwrap();
let sentinel: u64 = 0x0102_0304_0506_0708;
unsafe {
let ptr = s.usable_base() as *mut u64;
@@ -44,17 +44,17 @@ fn write_and_read_bottom_of_usable_region() {
#[test]
fn small_stack_allocates() {
assert!(Stack::new(4096).is_ok());
assert!(Stack::new(4096, 4096).is_ok());
}
#[test]
fn large_stack_allocates() {
assert!(Stack::new(8 * 1024 * 1024).is_ok());
assert!(Stack::new(8 * 1024 * 1024, 4096).is_ok());
}
#[test]
fn stack_size_at_least_requested() {
let s = Stack::new(64 * 1024).unwrap();
let s = Stack::new(64 * 1024, 4096).unwrap();
assert!(s.stack_size() >= 64 * 1024);
}
@@ -68,15 +68,28 @@ use std::process::Command;
fn run_as_child_if_requested() {
match env::var("SMARM_SUBTEST").as_deref() {
Ok("guard_page_direct") => {
let s = Stack::new(64 * 1024).unwrap();
let s = Stack::new(64 * 1024, 4096).unwrap();
unsafe {
let guard_ptr = s.usable_base().sub(1);
guard_ptr.write_volatile(0xAB);
}
std::process::exit(0);
}
Ok("wide_guard_top") => {
// One byte below the usable region, 64 KiB guard: must fault.
let s = Stack::new(64 * 1024, 64 * 1024).unwrap();
unsafe { s.usable_base().sub(1).write_volatile(0xAB); }
std::process::exit(0);
}
Ok("wide_guard_bottom") => {
// The very bottom page of a 64 KiB guard: an unprobed C-style
// leap over a small guard lands here — must still fault.
let s = Stack::new(64 * 1024, 64 * 1024).unwrap();
unsafe { s.usable_base().sub(64 * 1024).write_volatile(0xAB); }
std::process::exit(0);
}
Ok("stack_overflow") => {
let s = Stack::new(64 * 1024).unwrap();
let s = Stack::new(64 * 1024, 4096).unwrap();
unsafe {
let mut ptr = s.top().sub(1);
let stop = s.usable_base().sub(1);
@@ -121,3 +134,58 @@ fn stack_overflow_causes_sigsegv() {
assert_eq!(status.signal(), Some(11), "expected SIGSEGV, got: {:?}", status);
}
}
// ---------------------------------------------------------------------------
// RFC 019 — explicit shape: rounding, guard accessor, wide-guard coverage.
// ---------------------------------------------------------------------------
#[test]
fn sizes_round_up_to_page() {
let s = Stack::new(64 * 1024 + 1, 4096 + 1).unwrap();
assert_eq!(s.stack_size() % 4096, 0);
assert_eq!(s.guard_size() % 4096, 0);
assert!(s.stack_size() >= 64 * 1024 + 1);
assert!(s.guard_size() >= 4096 + 1);
}
#[test]
fn shape_reports_rounded_sizes() {
let s = Stack::new(64 * 1024, 64 * 1024).unwrap();
assert_eq!(s.shape(), (64 * 1024, 64 * 1024));
}
#[test]
fn usable_base_sits_above_guard() {
let s = Stack::new(64 * 1024, 64 * 1024).unwrap();
// The usable region must start exactly guard_size above the mapping
// base: a write at usable_base is legal, one byte below is not (the
// subprocess tests below prove the "not").
let sentinel: u64 = 0x1111_2222_3333_4444;
unsafe {
let ptr = s.usable_base() as *mut u64;
ptr.write_volatile(sentinel);
assert_eq!(ptr.read_volatile(), sentinel);
}
}
#[test]
fn wide_guard_faults_at_top() {
run_as_child_if_requested();
let status = spawn_subtest("wide_guard_top");
#[cfg(unix)]
{
use std::os::unix::process::ExitStatusExt;
assert_eq!(status.signal(), Some(11), "expected SIGSEGV, got: {:?}", status);
}
}
#[test]
fn wide_guard_faults_at_bottom() {
run_as_child_if_requested();
let status = spawn_subtest("wide_guard_bottom");
#[cfg(unix)]
{
use std::os::unix::process::ExitStatusExt;
assert_eq!(status.signal(), Some(11), "expected SIGSEGV, got: {:?}", status);
}
}
+141
View File
@@ -0,0 +1,141 @@
//! RFC 019 §7 — overflow diagnostics, observed from outside via subprocess
//! (mirrors tests/stack.rs's harness, plus stderr capture).
//!
//! Four cases:
//! - Rust recursion at defaults: probed frames walk into the guard →
//! tier-1 definitive message, death by SIGSEGV.
//! - FFI canary (96 KiB unprobed C local) at defaults: first touch lands
//! inside the 1 MiB guard → tier-1 message.
//! - FFI canary with the guard shrunk to 4 KiB: the frame steps over it
//! into unmapped VA below → tier-2 "stepped over" message. This is the
//! RFC's motivating incident (cargo-vendored gz build) reproduced.
//! - FFI canary with reserve raised to 256 KiB: fits, runs clean, exits 0 —
//! the §1 knob is the fix, proven by the same frame.
use std::env;
use std::process::Command;
unsafe extern "C" {
fn smarm_canary_burn();
}
/// Unbounded probed recursion; each frame dirties 4 KiB. black_box defeats
/// tail-call elision so the walk is real.
#[inline(never)]
#[allow(unconditional_recursion)]
fn recurse_forever(depth: u64) -> u64 {
let mut local = [0u8; 4096];
local[0] = depth as u8;
std::hint::black_box(&mut local);
recurse_forever(depth + 1).wrapping_add(local[0] as u64)
}
fn run_as_child_if_requested() {
let mode = match env::var("SMARM_DIAG_SUBTEST") {
Ok(m) => m,
Err(_) => return,
};
use smarm::runtime::Config;
use smarm::{spawn_with, SpawnOpts};
let rt = smarm::runtime::init(Config::exact(1));
rt.run(move || {
let opts = match mode.as_str() {
"rust_overflow" | "ffi_tier1" => SpawnOpts::default(),
// Small guard: the canary's 96 KiB displacement clears it.
"ffi_tier2" => SpawnOpts { guard_size: Some(4096), ..SpawnOpts::default() },
// Enough reserve: the same frame simply fits.
"ffi_clean" => SpawnOpts { stack_reserve: Some(256 * 1024), ..SpawnOpts::default() },
other => panic!("unknown subtest {other}"),
};
let is_rust = mode == "rust_overflow";
spawn_with(opts, move || {
if is_rust {
std::hint::black_box(recurse_forever(0));
} else {
unsafe { smarm_canary_burn() };
}
})
.join()
.unwrap();
});
std::process::exit(0);
}
fn spawn_subtest(name: &str) -> std::process::Output {
let exe = env::current_exe().unwrap();
Command::new(exe)
.env("SMARM_DIAG_SUBTEST", name)
.args(["--test-threads=1", "--quiet"])
.output()
.expect("failed to spawn subprocess")
}
#[cfg(unix)]
fn assert_died_sigsegv(out: &std::process::Output) {
use std::os::unix::process::ExitStatusExt;
assert_eq!(
out.status.signal(),
Some(11),
"expected death by SIGSEGV, got {:?}; stderr:\n{}",
out.status,
String::from_utf8_lossy(&out.stderr)
);
}
#[test]
fn rust_overflow_dies_with_tier1_message() {
run_as_child_if_requested();
let out = spawn_subtest("rust_overflow");
assert_died_sigsegv(&out);
let err = String::from_utf8_lossy(&out.stderr);
assert!(
err.contains("overflowed its stack") && err.contains("in the guard region"),
"missing tier-1 diagnostic; stderr:\n{err}"
);
assert!(err.contains("reserve=65536"), "wrong reserve in message:\n{err}");
assert!(err.contains("guard=1048576"), "wrong guard in message:\n{err}");
}
#[test]
fn ffi_canary_at_defaults_dies_with_tier1_message() {
run_as_child_if_requested();
let out = spawn_subtest("ffi_tier1");
assert_died_sigsegv(&out);
let err = String::from_utf8_lossy(&out.stderr);
// 96 KiB displacement from a 64 KiB reserve lands ~32 KiB into the
// 1 MiB guard: definitively classified.
assert!(
err.contains("in the guard region"),
"wide guard should catch the unprobed frame in tier 1; stderr:\n{err}"
);
}
#[test]
fn ffi_canary_over_small_guard_dies_with_tier2_message() {
run_as_child_if_requested();
let out = spawn_subtest("ffi_tier2");
assert_died_sigsegv(&out);
let err = String::from_utf8_lossy(&out.stderr);
assert!(
err.contains("stepped over it") && err.contains("below the guard"),
"expected tier-2 overshoot attribution; stderr:\n{err}"
);
assert!(err.contains("guard=4096"), "wrong guard in message:\n{err}");
}
#[test]
fn ffi_canary_with_enough_reserve_runs_clean() {
run_as_child_if_requested();
let out = spawn_subtest("ffi_clean");
assert!(
out.status.success(),
"canary should fit in 256 KiB reserve, got {:?}; stderr:\n{}",
out.status,
String::from_utf8_lossy(&out.stderr)
);
let err = String::from_utf8_lossy(&out.stderr);
assert!(
!err.contains("smarm: actor"),
"no diagnostic expected on the clean path; stderr:\n{err}"
);
}
+123
View File
@@ -0,0 +1,123 @@
//! RFC 019 commit 5 — pool recycle zaps a dead stack down to its retained
//! entry end, observed from the outside.
//!
//! A default-shaped stack that spiked deep and then died must not carry its
//! spike into the pool as resident RSS: `recycle_stack` DONTNEEDs everything
//! below the top `RECYCLE_RETAIN` bytes before pushing. The zap is
//! synchronous on the death path, so the drop is immediate — but the death
//! path itself races the observer's `join` return, hence the brief poll.
//!
//! Residency is measured with `mincore`, not smaps: a neighboring rw anon
//! mapping can land flush against the stack top and the kernel merges the
//! VMAs (observed under the full test run), so per-mapping smaps fields
//! over-count. The PROT_NONE guard below can never merge, so the usable
//! base is exactly the anchor VMA's start, and `mincore` counts pages
//! within [usable_base, usable_base + reserve) regardless of merging.
use smarm::runtime::{Config, RECYCLE_RETAIN};
use smarm::{channel, spawn, yield_now};
const RESERVE: usize = 4 * 1024 * 1024;
/// Burn ~`frames` × 4 KiB of stack, dirtying every frame.
#[inline(never)]
fn burn_stack(frames: usize) -> u64 {
let mut local = [0u8; 4096];
local[0] = frames as u8;
let below = if frames == 0 { 0 } else { burn_stack(frames - 1) };
std::hint::black_box(&mut local);
below.wrapping_add(local[0] as u64)
}
/// Resident-page count over [lo, lo + len) via mincore (len page-aligned).
fn resident_pages(lo: usize, len: usize) -> usize {
let page = 4096;
let mut vec = vec![0u8; len / page];
let ret = unsafe {
libc::mincore(lo as *mut libc::c_void, len, vec.as_mut_ptr())
};
assert_eq!(ret, 0, "mincore failed: {}", std::io::Error::last_os_error());
vec.iter().filter(|&&b| b & 1 != 0).count()
}
/// The [start, end) of the VMA containing `addr`.
fn vma_containing(addr: usize) -> (usize, usize) {
let maps = std::fs::read_to_string("/proc/self/maps").unwrap();
for line in maps.lines() {
if let Some((range, _)) = line.split_once(' ') {
if let Some((a, b)) = range.split_once('-') {
if let (Ok(start), Ok(end)) =
(usize::from_str_radix(a, 16), usize::from_str_radix(b, 16))
{
if start <= addr && addr < end {
return (start, end);
}
}
}
}
}
panic!("no VMA contains {addr:#x}");
}
fn vma_exists(addr: usize) -> bool {
let maps = std::fs::read_to_string("/proc/self/maps").unwrap();
for line in maps.lines() {
if let Some((range, _)) = line.split_once(' ') {
if let Some((a, b)) = range.split_once('-') {
if let (Ok(start), Ok(end)) =
(usize::from_str_radix(a, 16), usize::from_str_radix(b, 16))
{
if start <= addr && addr < end {
return true;
}
}
}
}
}
false
}
#[test]
fn recycle_zaps_dead_stack_down_to_retain() {
// Default reserve raised so the pool holds big stacks (default-shaped ⇒
// pooled) and the zap has something to bite; single scheduler.
let rt = smarm::runtime::init(Config::exact(1).stack_reserve(RESERVE));
rt.run(|| {
let (tx, rx) = channel::<usize>();
let h = spawn(move || {
let probe = 0u8;
let anchor = &probe as *const u8 as usize;
// The guard below is PROT_NONE and can never merge with the
// usable region, so the anchor VMA's start IS the usable base.
let (vlo, _) = vma_containing(anchor);
// Dirty ~3 MiB of the 4 MiB reserve, then die.
std::hint::black_box(burn_stack(768));
tx.send(vlo).unwrap();
});
let usable_base = rx.recv().unwrap();
h.join().unwrap();
// The zap span is everything below the retained entry end. DONTNEED
// on private anon discards synchronously and unconditionally, so
// this must go to exactly zero resident pages; the poll only covers
// the death path racing join's return.
let zap_len = RESERVE - RECYCLE_RETAIN;
let mut resident = usize::MAX;
for _ in 0..10_000 {
resident = resident_pages(usable_base, zap_len);
if resident == 0 {
break;
}
yield_now();
}
assert_eq!(
resident, 0,
"recycled stack's zap span still resident: {resident} pages in \
[{usable_base:#x}, +{zap_len:#x})"
);
// Pooled, not munmapped: the mapping must still be there.
assert!(vma_exists(usable_base), "default-shaped stack was unmapped instead of pooled");
});
}
+152
View File
@@ -0,0 +1,152 @@
//! RFC 019 commit 3 — park-path stack shrink, observed from the outside.
//!
//! The one integration-level claim of the shrink machinery: an actor that
//! spikes deep, returns shallow, and then parks past the cooldown gets its
//! dead span MADV_FREE'd — visible as `LazyFree` in `/proc/self/smaps`
//! within the stack's address range — while everything live survives.
//!
//! The high-water mark is *sampled* at context-save, so the spike yields
//! once at max depth to guarantee a sample there (in production, preemption
//! provides the quasi-random samples; a test must not rely on luck).
use smarm::runtime::{Config, SHRINK_COOLDOWN, SHRINK_THRESHOLD};
use smarm::{actor_info, channel, spawn, spawn_with, yield_now, ActorState, SpawnOpts};
/// Burn ~`frames` × 4 KiB of stack, yielding once at the bottom so the
/// context-save samples `sp` at max depth.
#[inline(never)]
fn burn_stack_yielding(frames: usize) -> u64 {
let mut local = [0u8; 4096];
local[0] = frames as u8;
let below = if frames == 0 {
yield_now();
0
} else {
burn_stack_yielding(frames - 1)
};
std::hint::black_box(&mut local);
below.wrapping_add(local[0] as u64)
}
/// Sum the `LazyFree:` kB of every smaps mapping intersecting [lo, hi).
fn lazy_free_bytes_in(lo: usize, hi: usize) -> usize {
let smaps = std::fs::read_to_string("/proc/self/smaps").unwrap();
let mut total_kb = 0usize;
let mut in_range = false;
for line in smaps.lines() {
if let Some((range, _)) = line.split_once(' ') {
if let Some((a, b)) = range.split_once('-') {
if let (Ok(start), Ok(end)) =
(usize::from_str_radix(a, 16), usize::from_str_radix(b, 16))
{
in_range = start < hi && end > lo;
continue;
}
}
}
if in_range {
if let Some(rest) = line.strip_prefix("LazyFree:") {
let kb: usize = rest.trim().trim_end_matches(" kB").trim().parse().unwrap();
total_kb += kb;
}
}
}
total_kb * 1024
}
#[test]
fn spike_then_parks_marks_lazyfree_and_keeps_live_data() {
// Single scheduler: the controller can gate on the worker being Parked.
let rt = smarm::runtime::init(Config::exact(1));
rt.run(|| {
let (park_tx, park_rx) = channel::<()>();
let (done_tx, done_rx) = channel::<(usize, u64)>();
let spike = 768 * 4096; // ~3 MiB, well past SHRINK_THRESHOLD
assert!(spike > SHRINK_THRESHOLD);
let worker = spawn_with(
SpawnOpts { stack_reserve: Some(8 * 1024 * 1024), ..SpawnOpts::default() },
move || {
// Live data that must survive the shrink, and an anchor
// address inside the stack for the smaps scan.
let live = [0xA5u8; 64];
let anchor = live.as_ptr() as usize;
// Spike: ~3 MiB deep, sampled at the bottom, unwound.
std::hint::black_box(burn_stack_yielding(768));
// Park past the cooldown. Each recv on the drained inbox is
// one park; the controller sends only when it sees us Parked.
for _ in 0..(SHRINK_COOLDOWN + 8) {
park_rx.recv().unwrap();
}
// Measure from inside: the stack spans ≤ 8 MiB below anchor.
let lazy = lazy_free_bytes_in(anchor - 8 * 1024 * 1024, anchor + 4096);
let checksum = live.iter().map(|&b| b as u64).sum();
done_tx.send((lazy, checksum)).unwrap();
},
);
let wpid = worker.pid();
for _ in 0..(SHRINK_COOLDOWN + 8) {
// Gate: send only once the worker is genuinely parked so every
// round is a real park-on-empty-mailbox.
loop {
match actor_info(wpid) {
Some(info) if info.state == ActorState::Parked => break,
Some(_) => yield_now(),
None => panic!("worker died early"),
}
}
park_tx.send(()).unwrap();
}
let (lazy, checksum) = done_rx.recv().unwrap();
// The spike was ~3 MiB; demand at least 2 MiB marked to leave slack
// for the redzone, rounding, and pages the unwind re-dirtied.
assert!(
lazy >= 2 * 1024 * 1024,
"expected ≥ 2 MiB LazyFree in the stack range, got {} bytes",
lazy
);
assert_eq!(checksum, 64 * 0xA5u64, "live stack data corrupted by shrink");
worker.join().unwrap();
});
}
/// Steady-state actors must never pay the syscall: an actor that parks a lot
/// but never spikes past the threshold ends with zero LazyFree in its stack.
#[test]
fn shallow_actor_never_shrinks() {
let rt = smarm::runtime::init(Config::exact(1));
rt.run(|| {
let (park_tx, park_rx) = channel::<()>();
let (done_tx, done_rx) = channel::<usize>();
let worker = spawn(move || {
let probe = 0u8;
let anchor = &probe as *const u8 as usize;
for _ in 0..(SHRINK_COOLDOWN + 8) {
park_rx.recv().unwrap();
}
done_tx.send(lazy_free_bytes_in(anchor - 64 * 1024, anchor + 4096)).unwrap();
});
let wpid = worker.pid();
for _ in 0..(SHRINK_COOLDOWN + 8) {
loop {
match actor_info(wpid) {
Some(info) if info.state == ActorState::Parked => break,
Some(_) => yield_now(),
None => panic!("worker died early"),
}
}
park_tx.send(()).unwrap();
}
assert_eq!(done_rx.recv().unwrap(), 0, "steady-state actor was shrunk");
worker.join().unwrap();
});
}