The slot table split (ROADMAP_v0.5 phase 2): slot lookup is lock-free, the
run path (yield/park/unpark/pop/resume) takes zero locks beyond the queue
mutex itself, and SharedState shrinks to { run_queue }.
Core pieces:
- src/raw_mutex.rs: hand-rolled 3-state futex mutex (Drepper mutex3),
non-poisoning; the guard enters NoPreempt, which both bounds hold times and
structurally closes the unwind-under-lock hole (the stop sentinel shares
the gate). Used for per-slot cold data, the free list, and the stack pool.
- Fixed slab Box<[Slot]> (default max_actors = 16_384, ~4 MiB, align(128)
against false sharing). Slots never move -> stable addresses, lock-free
index. Exhaustion panics loudly, naming Config::max_actors(n) as the fix.
Unbounded/segmented slab stays deferred (see ROADMAP).
- Per-slot AtomicU64 packing (generation << 32 | state), states
Vacant/Queued/Running/RunningNotified/Parked/Done. Every transition CASes
the packed word, so the generation check is atomic with the transition: no
ABA, no spurious unparks on recycled slots. RunningNotified replaces the
pending_unpark bool (the lost-wakeup window is a state, not a flag) and
uniformly fixes a LATENT LOST WAKEUP in the old Blocking-IO completion
path, which set the result for a still-Running actor without flagging it.
- Invariant: a pid is in the run queue at most once (pushes pair 1:1 with
transitions into Queued; only the scheduler does Queued->Running). This is
what makes phase 3's bounded rings sound.
- sp -> relaxed AtomicUsize; stop flag + first-resume closure as AtomicPtrs
(closure double-boxed for a thin pointer, swap-to-take): the resume path is
fully atomic.
- finalize_actor: Done published under the dying slot's cold lock (join's
check-or-register is linearized by it); link cascade locks peers ONE AT A
TIME (cold locks are leaves) with the acyclicity argument written at the
site; link() registers on the target first, then self (stale self-entries
are benign, every walk re-verifies the peer's word).
- Termination by counters: live_actors incremented in spawn pre-enqueue,
decremented at the very END of finalize after all wakeup enqueues; exit on
io_out == 0 (read before the queue lock, phase-1 ordering) && queue empty
&& live == 0. Soundness note at the site: any enqueue targets a live actor.
- spawn boxes the closure and acquires the stack BEFORE any runtime lock: no
allocation ever happens under a global lock anymore.
- with_runtime/try_with_runtime now enter NoPreempt for their full span.
This fixes a bug the rework exposed: install_actor allocated with
preemption enabled while the RUNTIME RefCell borrow was live; a timeslice
preemption there migrates the actor across OS threads and the borrow guard
increments one thread's RefCell count and decrements another's — underflow
to 'permanently mutably borrowed', cascading panics (caught by stress
suite: deterministic non-unwinding-panic abort in lost_wakeup_many_pairs).
The old code was safe only by accident; now it's structural.
- Behavior note: request_stop on a RUNNING target now marks it
RunningNotified, so its next park returns immediately to an observation
point — faster stop observation; parked/queued/done semantics unchanged.
Validated: 22 suites green in release (1/2/8-thread oversubscribed on the
1-core sandbox) and in debug with all debug_asserts live; stress suite x5.
187 lines
7.5 KiB
Rust
187 lines
7.5 KiB
Rust
//! Process links + `trap_exit`.
|
|
//!
|
|
//! A *link* is bidirectional and persistent, in contrast to a [`monitor`]
|
|
//! (unidirectional, one-shot): once two actors are linked, the *abnormal*
|
|
//! death of either propagates to the other. "Abnormal" means a panic or a
|
|
//! cooperative [`request_stop`] — a normal return never propagates.
|
|
//!
|
|
//! What propagation *does* depends on whether the surviving peer traps exits:
|
|
//!
|
|
//! - **not trapping** (the default): the peer is cooperatively
|
|
//! [`request_stop`]'d, so a crash fate-shares across the whole link set —
|
|
//! Erlang's "let it crash". Because the peer's own links are walked when
|
|
//! *it* finalizes, the stop cascades transitively.
|
|
//! - **trapping** (called [`trap_exit`]): the peer instead receives an
|
|
//! [`ExitSignal`] *message* on its trap inbox and keeps running, turning
|
|
//! peer failures into ordinary inbox events (Erlang's
|
|
//! `process_flag(trap_exit, true)`).
|
|
//!
|
|
//! Linking an already-dead pid is not a silent no-op: it behaves exactly like
|
|
//! an immediate abnormal death of the target with reason [`DownReason::NoProc`]
|
|
//! — a trapping caller gets a message, a non-trapping caller is stopped.
|
|
//!
|
|
//! ## Payload
|
|
//!
|
|
//! [`ExitSignal`] reuses [`DownReason`] and, like a monitor, does **not**
|
|
//! carry the panic payload: a panic's payload has a single owner and is
|
|
//! delivered to whoever `join()`s the actor. A trap inbox only learns *that*
|
|
//! (and *how*) a peer died.
|
|
//!
|
|
//! ## Inbox
|
|
//!
|
|
//! The trap inbox is a dedicated channel, distinct from the monitor [`Down`]
|
|
//! channel: monitors are documented as one-shot/unidirectional, whereas a trap
|
|
//! inbox receives one [`ExitSignal`] per linked peer death over the actor's
|
|
//! lifetime. [`trap_exit`] enables trapping and hands back the inbox in one
|
|
//! call; calling it again installs a fresh inbox (the previous one closes).
|
|
//!
|
|
//! ## Races
|
|
//!
|
|
//! [`link`] registration and `finalize_actor` (in `runtime`) both serialize on
|
|
//! the shared mutex, so a target that is live when the link is registered is
|
|
//! guaranteed to propagate its eventual death; there is no window in which the
|
|
//! death slips between the liveness check and the registration. As elsewhere,
|
|
//! we never `send` or `request_stop` while holding the shared lock — those
|
|
//! re-enter the runtime — so the act is always performed after the lock is
|
|
//! released.
|
|
//!
|
|
//! [`monitor`]: crate::monitor::monitor
|
|
//! [`Down`]: crate::monitor::Down
|
|
//! [`request_stop`]: crate::scheduler::request_stop
|
|
|
|
use crate::channel::{channel, Receiver};
|
|
use crate::monitor::DownReason;
|
|
use crate::pid::Pid;
|
|
use crate::scheduler::{request_stop, self_pid, with_runtime};
|
|
|
|
/// A linked peer's death, delivered to a trapping actor's inbox.
|
|
///
|
|
/// `Copy` because it carries no payload — see the module docs for why the
|
|
/// panic payload is *not* included.
|
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
|
pub struct ExitSignal {
|
|
/// The linked pid that died (or the dead pid that was linked).
|
|
pub from: Pid,
|
|
/// How it went down. Only ever `Panic`, `Stopped`, or `NoProc` — a normal
|
|
/// `Exit` does not propagate, so it never appears here.
|
|
pub reason: DownReason,
|
|
}
|
|
|
|
/// Trap exits on the current actor and return its exit inbox.
|
|
///
|
|
/// After this call, the abnormal death of a linked peer arrives here as an
|
|
/// [`ExitSignal`] message instead of cooperatively stopping this actor.
|
|
/// Calling it again installs a fresh inbox; the previously returned receiver
|
|
/// then closes.
|
|
pub fn trap_exit() -> Receiver<ExitSignal> {
|
|
let (tx, rx) = channel::<ExitSignal>();
|
|
let me = self_pid();
|
|
with_runtime(|inner| {
|
|
if let Some(slot) = inner.slot_at(me) {
|
|
let mut cold = slot.cold.lock();
|
|
// Own slot: generation is necessarily current (we're running).
|
|
if let Some(actor) = cold.actor.as_mut() {
|
|
actor.trap = Some(tx);
|
|
}
|
|
}
|
|
});
|
|
rx
|
|
}
|
|
|
|
/// Bidirectionally link the current actor to `target`.
|
|
///
|
|
/// If `target` is live, the link is recorded on both actors and either's
|
|
/// abnormal death will propagate to the other. If `target` is already gone,
|
|
/// this delivers an immediate [`DownReason::NoProc`] exit signal to the caller
|
|
/// (a message if trapping, otherwise a cooperative stop). Linking yourself, or
|
|
/// re-linking an existing peer, is a no-op.
|
|
pub fn link(target: Pid) {
|
|
let me = self_pid();
|
|
if target == me {
|
|
return;
|
|
}
|
|
|
|
// Cold locks are leaves: never hold two at once. The link is recorded one
|
|
// side at a time, TARGET FIRST — that ordering is what makes the race
|
|
// window sound:
|
|
//
|
|
// - Once `me` is in `target.links`, the target's death always reaches us
|
|
// (its finalize cascade walks that list). So after step 1 succeeds, the
|
|
// link semantics are already live.
|
|
// - If the target dies between step 1 and step 2, its cascade removes
|
|
// `target` from OUR links (a no-op, we haven't added it yet) and
|
|
// delivers the exit signal — correct, the link was established. Our
|
|
// subsequent step-2 insert leaves a stale `target` entry in `me.links`;
|
|
// stale entries are benign by construction (every cascade walk
|
|
// re-verifies the peer's word; `unlink` removes them like any other).
|
|
//
|
|
// The reverse order would be unsound: target dying in the window would
|
|
// walk its links WITHOUT us — a silently dead link that we believe is live.
|
|
let registered_on_target = with_runtime(|inner| match inner.slot_at(target) {
|
|
Some(slot) => {
|
|
let mut cold = slot.cold.lock();
|
|
if slot.is_live_for(target) && cold.actor.is_some() {
|
|
if !cold.links.contains(&me) {
|
|
cold.links.push(me);
|
|
}
|
|
true
|
|
} else {
|
|
false
|
|
}
|
|
}
|
|
None => false,
|
|
});
|
|
|
|
if registered_on_target {
|
|
with_runtime(|inner| {
|
|
let slot = inner.slot_at(me).expect("link: own slot vanished");
|
|
let mut cold = slot.cold.lock();
|
|
if !cold.links.contains(&target) {
|
|
cold.links.push(target);
|
|
}
|
|
});
|
|
return;
|
|
}
|
|
|
|
// Target already gone: deliver NoProc to ourselves — as a message if
|
|
// trapping, otherwise as a cooperative stop.
|
|
let my_trap = with_runtime(|inner| {
|
|
inner.slot_at(me).and_then(|slot| {
|
|
let cold = slot.cold.lock();
|
|
cold.actor.as_ref().and_then(|a| a.trap.clone())
|
|
})
|
|
});
|
|
match my_trap {
|
|
Some(tx) => {
|
|
let _ = tx.send(ExitSignal { from: target, reason: DownReason::NoProc });
|
|
}
|
|
None => request_stop(me),
|
|
}
|
|
}
|
|
|
|
/// Remove the link between the current actor and `target`, in both directions.
|
|
///
|
|
/// After this, neither actor's death propagates to the other. A no-op if the
|
|
/// two were not linked.
|
|
pub fn unlink(target: Pid) {
|
|
let me = self_pid();
|
|
if target == me {
|
|
return;
|
|
}
|
|
with_runtime(|inner| {
|
|
// One cold lock at a time (leaf rule). Order is immaterial here:
|
|
// a half-removed link is just a stale entry on one side, and stale
|
|
// entries are benign (re-verified on every cascade walk).
|
|
if let Some(slot) = inner.slot_at(me) {
|
|
let mut cold = slot.cold.lock();
|
|
cold.links.retain(|p| *p != target);
|
|
}
|
|
if let Some(slot) = inner.slot_at(target) {
|
|
let mut cold = slot.cold.lock();
|
|
if slot.generation() == target.generation() {
|
|
cold.links.retain(|p| *p != me);
|
|
}
|
|
}
|
|
});
|
|
}
|