From 64bd7e81a5c4a189e0856d969b9892c76361af69 Mon Sep 17 00:00:00 2001 From: smarm-agent Date: Mon, 15 Jun 2026 11:21:09 +0000 Subject: [PATCH] docs: neutralize excised-spin references in per-switch N=1 profile The profile was captured against the spin-enabled build. With the RFC 004 spinning experiment excised from master, the ~12% hot-path futex_wake it attributed to the spinning submit-rule no longer exists (idle wait is back to thread::sleep). Add a provenance note, mark the futex_wake row/finding as excised spin machinery, and drop the now-moot 'gate the submit wake' lead. Spin-independent findings (shims ~1%, schedule_loop + run-queue ~33%) unchanged. --- docs/perswitch-profile-n1.md | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/docs/perswitch-profile-n1.md b/docs/perswitch-profile-n1.md index ab1527b..8740055 100644 --- a/docs/perswitch-profile-n1.md +++ b/docs/perswitch-profile-n1.md @@ -6,6 +6,15 @@ and no cross-core traffic). Sandbox: 1 core, kernel 6.18, **no PMU** (hardware counters unavailable), so attribution is from `perf record -e task-clock` (software timer sampling) plus the bench's own rdtsc/wall brackets. +> **Provenance (post-excision).** This profile was captured against the +> spin-enabled build (the pre-excision HEAD, with the RFC 004 spinning workers +> live in `src/runtime.rs`). The RFC 004 spinning experiment has since been +> excised from `master`; idle schedulers are back to the historical +> `thread::sleep` wait. The `futex_wake` attribution below therefore reflects +> spinning machinery that is **no longer present on current master** — see the +> per-row and per-finding notes. The shim, `schedule_loop`, and run-queue +> findings are spin-independent and remain valid. + ## Numbers (stable across runs) - Round-trip p50 ≈ **303 ns ≈ 828 cyc** (instrumentation floor subtracted). @@ -20,7 +29,7 @@ counters unavailable), so attribution is from `perf record -e task-clock` |------:|--------|--------| | 24.8% | `runtime::schedule_loop` | scheduler logic (slot-word/epoch + dispatch) | | 8.6% | `MutexQueue::push`/`pop`/`len` | run-queue ops | -| ~12% | `do_syscall_64`+`syscall`+`futex_*` | **futex_wake on the hot path** | +| ~12% | `do_syscall_64`+`syscall`+`futex_*` | **futex_wake on the hot path** — spinning submit-rule wake; removed by the RFC 004 excision (not on current master) | | 3.5% | `IoThread::drain_completions` | the always-on IO thread (`run()` starts one) | | ~30% | `main` + `clock_gettime`/Timespec + `quicksort` | **instrumentation** (timing + percentile sort) | | ~1% | `switch_to_scheduler`+`switch_to_actor_asm`+ sp accessors | **the context shims + TLS** | @@ -34,14 +43,13 @@ self-time.** The N=1 cost is dominated by: 1. **`schedule_loop` + run-queue ops (~33%)** — the epoch/slot-word transition and the mutex run-queue push/pop on every re-queue. -2. **A `futex_wake` syscall (~12%)** firing on the hot path even though nothing - is parked. This is the **submit-rule wake** the handoff itself flagged (RFC - 004 finding #1: "a parallelism/latency optimisation, NOT a liveness guard"). - In a single-scheduler always-runnable loop it is pure cost — no one is ever - parked to wake. **First cheap-win candidate: suppress the submit wake when - there is no parked waiter** (a parked-count / n_spinning check before the - syscall). Needs care — finding #2's AllDone broadcast is the liveness-critical - one and must NOT be touched. +2. **A `futex_wake` syscall (~12%)** fired on the hot path even though nothing + was parked. This was the spinning **submit-rule wake** introduced by the RFC + 004 experiment — a parallelism/latency optimisation, not a liveness guard. In + a single-scheduler always-runnable loop it was pure cost (no one was ever + parked to wake). The RFC 004 excision removed this wake with the rest of the + spinning machinery: on current master idle schedulers use `thread::sleep` + again, so the N=1 hot path no longer makes this syscall. ## What this does and does NOT show @@ -51,7 +59,9 @@ self-time.** The N=1 cost is dominated by: that is observable at N=1 on one core.** This profile does NOT refute it; it establishes that the shim is cheap *until cores contend*. - So the spike question sharpens into two separable costs: - - **N=1 floor:** scheduler logic + a likely-removable futex_wake. Actionable now. + - **N=1 floor:** scheduler logic (`schedule_loop` + run-queue ops). The + futex_wake component was spinning machinery and is gone post-excision, so the + remaining N=1 floor is the scheduler core itself. - **N→8 slope:** the shim/TLS/coherency cost. Needs the many-core box + a `remote` bench mode (wake straddling two schedulers) + hardware PMU counters (cache-misses, `MEM_LOAD…HITM` for coherency) — none available in this sandbox.