preempt: explicit check!() macro for no-alloc loops

Stable Rust emits stack probes inline (subq/movq/jne loop) rather than
calling __rust_probestack, so there's no transparent hook for stack-
frame preemption. Override of __rust_probestack links cleanly but never
runs. Falling back to an explicit check!() that users drop into hot
compute loops.

check!() decrements the same ALLOC_COUNT counter as the heap path, so
both event sources fire timeslice checks at the same rate. Documents
the prep-to-park invariant on maybe_preempt — library code that
registers a wakeup and then parks must keep that window alloc-free and
check-free, or a preemption-driven yield in the middle would lose the
wakeup.
This commit is contained in:
Claude
2026-05-22 05:37:04 +00:00
parent 51bfccc3c2
commit d3ab81b833
3 changed files with 109 additions and 7 deletions

View File

@@ -42,3 +42,25 @@ pub use scheduler::{
block_on_io, run, self_pid, sleep, spawn, spawn_under, yield_now, JoinError, JoinHandle,
};
pub use supervisor::Signal;
// ---------------------------------------------------------------------------
// check!() — explicit preemption point for tight no-alloc loops.
// ---------------------------------------------------------------------------
/// Voluntarily check whether this actor's timeslice has expired, yielding
/// if so. Drop this into hot compute loops that don't allocate (heap or
/// large stack frames) — without it, such loops monopolise the scheduler
/// until they return.
///
/// Decrements the same per-actor event counter as the heap allocator's
/// preemption hook, so the check rate is identical regardless of whether
/// the actor is alloc-heavy, check-heavy, or mixed.
///
/// No-op outside an actor (the runtime's `PREEMPTION_ENABLED` flag is
/// false there).
#[macro_export]
macro_rules! check {
() => {
$crate::preempt::maybe_preempt()
};
}

View File

@@ -6,10 +6,16 @@
//! `switch_to_scheduler` to yield. Resetting the counter to `ALLOC_INTERVAL`
//! amortises the RDTSC across many cheap events.
//!
//! Events today are heap allocations (via `PreemptingAllocator`). v0.2 will
//! add stack-frame entries as a second event source — frames are stack
//! allocations, the counter naming still fits — sharing this same counter
//! so both routes behave consistently.
//! Two event sources today:
//! - `PreemptingAllocator` — heap allocations.
//! - `smarm::check!()` — explicit preemption point for tight no-alloc
//! loops, since stable Rust gives us no transparent way to preempt
//! such loops (`__rust_probestack` is emitted inline by LLVM and not
//! called at runtime).
//!
//! Both sources share `ALLOC_COUNT`, so the timeslice check fires at the
//! same rate regardless of whether the actor is alloc-heavy, check-heavy,
//! or mixed.
//!
//! All state is thread-local. The scheduler enables preemption on resume
//! and disables it on the return path, so the scheduler can never preempt
@@ -80,9 +86,17 @@ unsafe impl GlobalAlloc for PreemptingAllocator {
}
/// Shared preemption check. Called by every preemption event source — the
/// heap allocator today, the stack-frame entry hook in v0.2. Decrements
/// `ALLOC_COUNT`; every `ALLOC_INTERVAL` calls reads the timeslice clock
/// and yields if expired.
/// heap allocator today, `smarm::check!()` for tight no-alloc loops.
/// Decrements `ALLOC_COUNT`; every `ALLOC_INTERVAL` calls reads the
/// timeslice clock and yields if expired.
///
/// **Invariant**: must not be called inside a "prep-to-park" region —
/// e.g. between registering as a channel's parked receiver and calling
/// `park_current()`. A preemption-driven yield in that window would
/// reach the scheduler with state=Runnable, the unparker would no-op,
/// the actor would then park, and the wakeup would be lost. Library
/// code that touches the parking primitives must keep its prep-to-park
/// regions allocation-free and check!()-free.
#[inline(always)]
pub fn maybe_preempt() {
ALLOC_COUNT.with(|c| {