perf(channel): capture the receiver's runtime Weak once per channel, not once per park
Upstream's off-runtime wake (1002777) captured a Weak<RuntimeInner> in the
parked_receiver tuple on every park, so every park/unpark round-trip paid an
Arc::downgrade plus the matching Weak drop — a locked RMW pair on one globally
shared counter, on the hot path of every channel workload.
The receiver never migrates between runtimes, so one capture is enough: the
Weak moves into Inner<T>, filled in on first park (a branch on an Option
thereafter), and unpark_at_via takes a closure so the fallback only re-locks
and clones off a scheduler thread, where the in-runtime fast path has already
declined. runtime_weak() returns Option and no longer panics off-runtime.
Sandbox 1T channel steady round-trip: 290 -> 270 ns (upstream's shape measured
427 -> 445 on its own base). Full suite green under reltest, including
upstream's tests/cross_thread_wake.rs.
This commit is contained in:
+19
-8
@@ -595,12 +595,16 @@ pub(crate) fn unpark_at(pid: Pid, epoch: u32) {
|
||||
}
|
||||
|
||||
// The current actor's runtime as a `Weak`, for a waker that must reach the
|
||||
// runtime from a foreign thread later. A channel captures this when its
|
||||
// receiver parks, so a cross-thread `send` can wake without the `RUNTIME`
|
||||
// thread-local (unset off a scheduler thread). Panics outside `Runtime::run()`,
|
||||
// the same contract as `begin_wait`.
|
||||
pub(crate) fn runtime_weak() -> Weak<RuntimeInner> {
|
||||
with_runtime(Arc::downgrade)
|
||||
// runtime from a foreign thread later. A channel captures this ONCE, the first
|
||||
// time its receiver parks, so a cross-thread `send` can wake without the
|
||||
// `RUNTIME` thread-local (unset off a scheduler thread). `None` off a
|
||||
// scheduler thread, where there is nothing to capture.
|
||||
//
|
||||
// Deliberately not called per park: `Arc::downgrade` plus the matching drop is
|
||||
// a locked RMW pair on one globally shared counter, and the park/unpark
|
||||
// round-trip is the hot path of every channel workload.
|
||||
pub(crate) fn runtime_weak() -> Option<Weak<RuntimeInner>> {
|
||||
try_with_runtime(Arc::downgrade)
|
||||
}
|
||||
|
||||
// Epoch-matched wake of `pid` from a waker that may or may not be on a
|
||||
@@ -608,11 +612,18 @@ pub(crate) fn runtime_weak() -> Weak<RuntimeInner> {
|
||||
// (preemption-gated, slot-eligible); off one that path is a silent no-op, so
|
||||
// we reach the runtime through `rt` — the `Weak` the waker captured while it
|
||||
// was in-runtime. Mirrors the IO backend's cross-context wake (io.rs, RFC 018).
|
||||
pub(crate) fn unpark_at_via(pid: Pid, epoch: u32, rt: &Weak<RuntimeInner>) {
|
||||
pub(crate) fn unpark_at_via(
|
||||
pid: Pid,
|
||||
epoch: u32,
|
||||
rt: impl FnOnce() -> Option<Weak<RuntimeInner>>,
|
||||
) {
|
||||
if try_with_runtime(|inner| inner.unpark_at(pid, epoch)).is_some() {
|
||||
return;
|
||||
}
|
||||
if let Some(inner) = rt.upgrade() {
|
||||
// Off a scheduler thread only: `rt` re-takes the waker's lock to read the
|
||||
// captured `Weak`, which is why it is a closure and not a value — the
|
||||
// in-runtime path above must not pay for it.
|
||||
if let Some(inner) = rt().as_ref().and_then(Weak::upgrade) {
|
||||
inner.unpark_at(pid, epoch);
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user