The slot table split (ROADMAP_v0.5 phase 2): slot lookup is lock-free, the
run path (yield/park/unpark/pop/resume) takes zero locks beyond the queue
mutex itself, and SharedState shrinks to { run_queue }.
Core pieces:
- src/raw_mutex.rs: hand-rolled 3-state futex mutex (Drepper mutex3),
non-poisoning; the guard enters NoPreempt, which both bounds hold times and
structurally closes the unwind-under-lock hole (the stop sentinel shares
the gate). Used for per-slot cold data, the free list, and the stack pool.
- Fixed slab Box<[Slot]> (default max_actors = 16_384, ~4 MiB, align(128)
against false sharing). Slots never move -> stable addresses, lock-free
index. Exhaustion panics loudly, naming Config::max_actors(n) as the fix.
Unbounded/segmented slab stays deferred (see ROADMAP).
- Per-slot AtomicU64 packing (generation << 32 | state), states
Vacant/Queued/Running/RunningNotified/Parked/Done. Every transition CASes
the packed word, so the generation check is atomic with the transition: no
ABA, no spurious unparks on recycled slots. RunningNotified replaces the
pending_unpark bool (the lost-wakeup window is a state, not a flag) and
uniformly fixes a LATENT LOST WAKEUP in the old Blocking-IO completion
path, which set the result for a still-Running actor without flagging it.
- Invariant: a pid is in the run queue at most once (pushes pair 1:1 with
transitions into Queued; only the scheduler does Queued->Running). This is
what makes phase 3's bounded rings sound.
- sp -> relaxed AtomicUsize; stop flag + first-resume closure as AtomicPtrs
(closure double-boxed for a thin pointer, swap-to-take): the resume path is
fully atomic.
- finalize_actor: Done published under the dying slot's cold lock (join's
check-or-register is linearized by it); link cascade locks peers ONE AT A
TIME (cold locks are leaves) with the acyclicity argument written at the
site; link() registers on the target first, then self (stale self-entries
are benign, every walk re-verifies the peer's word).
- Termination by counters: live_actors incremented in spawn pre-enqueue,
decremented at the very END of finalize after all wakeup enqueues; exit on
io_out == 0 (read before the queue lock, phase-1 ordering) && queue empty
&& live == 0. Soundness note at the site: any enqueue targets a live actor.
- spawn boxes the closure and acquires the stack BEFORE any runtime lock: no
allocation ever happens under a global lock anymore.
- with_runtime/try_with_runtime now enter NoPreempt for their full span.
This fixes a bug the rework exposed: install_actor allocated with
preemption enabled while the RUNTIME RefCell borrow was live; a timeslice
preemption there migrates the actor across OS threads and the borrow guard
increments one thread's RefCell count and decrements another's — underflow
to 'permanently mutably borrowed', cascading panics (caught by stress
suite: deterministic non-unwinding-panic abort in lost_wakeup_many_pairs).
The old code was safe only by accident; now it's structural.
- Behavior note: request_stop on a RUNNING target now marks it
RunningNotified, so its next park returns immediately to an observation
point — faster stop observation; parked/queued/done semantics unchanged.
Validated: 22 suites green in release (1/2/8-thread oversubscribed on the
1-core sandbox) and in debug with all debug_asserts live; stress suite x5.
145 lines
5.9 KiB
Rust
145 lines
5.9 KiB
Rust
//! Actor descriptor and trampoline.
|
|
//!
|
|
//! An `Actor` owns its stack and holds the closure it will run. The
|
|
//! `trampoline` is a fixed `extern "C-unwind" fn()` that every actor enters
|
|
//! through; it pulls the closure out of a thread-local set by the scheduler
|
|
//! immediately before resume, invokes it inside `catch_unwind`, records the
|
|
//! outcome, and switches back to the scheduler.
|
|
//!
|
|
//! Why a thread-local and not, say, passing the closure pointer via a
|
|
//! register? Because the first resume goes through `ret`, not `call`, and
|
|
//! we have no other channel for parameters. The scheduler sets the
|
|
//! thread-local, switches in, the trampoline reads it. After the first
|
|
//! resume the closure has been consumed, so subsequent resumes don't need it.
|
|
|
|
use crate::context::switch_to_scheduler;
|
|
use crate::pid::Pid;
|
|
use crate::stack::Stack;
|
|
use std::any::Any;
|
|
use std::cell::{Cell, RefCell};
|
|
use std::panic;
|
|
|
|
/// What an actor produced when it finished. Stored on the actor's slot,
|
|
/// drained by `JoinHandle::join` once the slot is marked done.
|
|
pub enum Outcome {
|
|
Exit,
|
|
Panic(Box<dyn Any + Send>),
|
|
/// The actor was cooperatively cancelled via `request_stop`: a sentinel
|
|
/// panic unwound its stack (running Drop) and the trampoline recognised
|
|
/// the sentinel. Distinct from a user `Panic` — there is no payload to
|
|
/// propagate.
|
|
Stopped,
|
|
}
|
|
|
|
/// The payload of the sentinel panic raised at an observation point when an
|
|
/// actor has been flagged for cooperative cancellation. Zero-sized; its only
|
|
/// job is to be recognisable in the trampoline's `catch_unwind` so the
|
|
/// teardown is reported as `Outcome::Stopped` rather than `Outcome::Panic`.
|
|
///
|
|
/// User code that wraps its own `catch_unwind` can swallow this (cf. Erlang's
|
|
/// `catch`); the stop flag stays set, so the next observation point re-raises.
|
|
pub(crate) struct StopSentinel;
|
|
|
|
// Thread-locals that the scheduler writes immediately before `switch_to_actor`.
|
|
thread_local! {
|
|
/// The closure for the actor we're about to resume *for the first time*.
|
|
/// Consumed on first entry into the trampoline; `None` thereafter.
|
|
static CURRENT_ACTOR_BOX: RefCell<Option<Box<dyn FnOnce() + Send>>> =
|
|
const { RefCell::new(None) };
|
|
|
|
/// The PID of the actor currently executing on this OS thread.
|
|
/// Set on every resume so that `self_pid()` works inside actor code.
|
|
static CURRENT_PID: Cell<Option<Pid>> = const { Cell::new(None) };
|
|
|
|
/// Filled by the trampoline when the actor returns (normally or via
|
|
/// panic). The scheduler reads this after `switch_to_actor` returns.
|
|
static LAST_OUTCOME: RefCell<Option<Outcome>> = const { RefCell::new(None) };
|
|
|
|
/// Set by the trampoline on completion; reset by the scheduler before
|
|
/// each resume so it never sees stale state.
|
|
static ACTOR_DONE: Cell<bool> = const { Cell::new(false) };
|
|
}
|
|
|
|
pub fn set_current_actor_box(b: Box<dyn FnOnce() + Send>) {
|
|
CURRENT_ACTOR_BOX.with(|c| *c.borrow_mut() = Some(b));
|
|
}
|
|
|
|
pub fn set_current_pid(p: Pid) {
|
|
CURRENT_PID.with(|c| c.set(Some(p)));
|
|
}
|
|
|
|
pub fn clear_current_pid() {
|
|
CURRENT_PID.with(|c| c.set(None));
|
|
}
|
|
|
|
pub fn current_pid() -> Option<Pid> {
|
|
CURRENT_PID.with(|c| c.get())
|
|
}
|
|
|
|
pub fn reset_actor_done() {
|
|
ACTOR_DONE.with(|c| c.set(false));
|
|
}
|
|
|
|
pub fn is_actor_done() -> bool {
|
|
ACTOR_DONE.with(|c| c.get())
|
|
}
|
|
|
|
pub fn take_last_outcome() -> Option<Outcome> {
|
|
LAST_OUTCOME.with(|r| r.borrow_mut().take())
|
|
}
|
|
|
|
/// The function whose address is written as the `ret` target on every actor
|
|
/// stack. The compiler must not inline this away. `extern "C-unwind"` permits
|
|
/// unwinding to cross the boundary, but `catch_unwind` here means unwinding
|
|
/// never actually does.
|
|
pub extern "C-unwind" fn trampoline() {
|
|
let b = CURRENT_ACTOR_BOX.with(|c| c.borrow_mut().take())
|
|
.expect("trampoline entered without a closure set");
|
|
|
|
let outcome = match panic::catch_unwind(panic::AssertUnwindSafe(b)) {
|
|
Ok(()) => Outcome::Exit,
|
|
Err(payload) => {
|
|
if payload.is::<StopSentinel>() {
|
|
Outcome::Stopped
|
|
} else {
|
|
Outcome::Panic(payload)
|
|
}
|
|
}
|
|
};
|
|
|
|
LAST_OUTCOME.with(|r| *r.borrow_mut() = Some(outcome));
|
|
ACTOR_DONE.with(|c| c.set(true));
|
|
|
|
// Hand control back. The scheduler will tear down our slot and never
|
|
// resume us again.
|
|
unsafe { switch_to_scheduler() };
|
|
// Unreachable. If it isn't, the scheduler has a bug.
|
|
unreachable!("scheduler resumed a done actor");
|
|
}
|
|
|
|
/// One actor's worth of state. Owned by the scheduler's slot table.
|
|
pub struct Actor {
|
|
/// The PID this actor was assigned at spawn time.
|
|
pub pid: Pid,
|
|
/// The stack the actor runs on. Dropped (munmap'd) when the actor dies.
|
|
/// (The saved stack pointer lives on the `Slot` as an atomic, not here:
|
|
/// it is hot scheduling state, read/written without the cold lock.)
|
|
pub stack: Stack,
|
|
/// The PID of this actor's supervisor. Used to deliver `Signal` on death.
|
|
pub supervisor: Pid,
|
|
/// Cooperative-cancellation flag. `request_stop` sets it (and unparks a
|
|
/// parked actor); the actor observes it lock-free at its next observation
|
|
/// point — see `preempt::check_cancelled`. Shared via `Arc` so the running
|
|
/// actor can poll it without taking the shared mutex. Lives on the `Actor`
|
|
/// (constructed fresh at spawn), so unlike a `Slot` field it needs no reset
|
|
/// in the slot-recycling paths.
|
|
pub stop: std::sync::Arc<std::sync::atomic::AtomicBool>,
|
|
/// Trap-exit inbox (roadmap #3). `None` until the actor calls
|
|
/// `link::trap_exit()`; `Some(tx)` means an abnormal death of a linked
|
|
/// peer is delivered here as an `ExitSignal` *message* instead of
|
|
/// cooperatively stopping this actor. Lives on the `Actor` (fresh per
|
|
/// spawn) for the same reason as `stop`: no slot-recycling reset needed,
|
|
/// and a restarted child correctly starts out *not* trapping.
|
|
pub trap: Option<crate::channel::Sender<crate::link::ExitSignal>>,
|
|
}
|