benches: document spin_sweep in README (catalog + RFC 004 section, knobs, 1-core caveat); add vs-tokio summary to sweep.py. print_results emits a human 'vs tokio' block + greppable VSTOKIO,<bench>,<smarm_label>,<smarm_us>,<tokio_label>,<tokio_us>,<ratio>,<winner> lines, paired like-for-like by thread count (single vs current_thread; multi vs multi; multi_thread_scaling per matching count). ratio = tokio/smarm so >1 = smarm faster. Also gitignore __pycache__/*.pyc.
This commit is contained in:
smarm-agent
@@ -2,3 +2,5 @@ target
|
||||
Cargo.lock
|
||||
smarm_trace.json
|
||||
/bench_results/
|
||||
__pycache__/
|
||||
*.pyc
|
||||
|
||||
@@ -21,6 +21,7 @@ cargo bench # all of them (slow; rarely what you want)
|
||||
| `tokio_favored.rs` | Workloads tokio's model is built for. Expect to lose; the value is knowing *by how much* and catching the gap widening. |
|
||||
| `rq_micro.rs` | Run-queue **structures** in isolation (no runtime, no actors): push/pop throughput sweeping thread count × producer:consumer ratio. Covers all three queue types in one binary — the types compile in every build; only the runtime's alias is feature-selected. |
|
||||
| `rq_runtime.rs` | The **whole scheduler** with the compile-time-selected queue: yield-storm (pure queue churn), ping-pong-pairs (park/unpark latency), spawn-storm (slab + free list + queue churn), sweeping scheduler count. Comparing variants requires rebuilding per `rq-*` feature. |
|
||||
| `spin_sweep.rs` | RFC 004 spinning workers: wake **latency vs idle-CPU** over `spin_budget_cycles`. Four workloads (remote-wake, fork-join, half-load, pure-idle) × budget × `max_spinners` × scheduler count; latency percentiles plus cores-busy from `getrusage`. The data that picks the budget default and on/off posture. |
|
||||
|
||||
## The run-queue shootout
|
||||
|
||||
@@ -55,6 +56,70 @@ cargo bench --bench rq_runtime --no-default-features --features rq-striped
|
||||
| `SMARM_BENCH_PAIRS` / `_ROUNDTRIPS` | `32` / `1000` | `rq_runtime` ping-pong |
|
||||
| `SMARM_BENCH_SPAWNS` | `5000` | `rq_runtime` spawn-storm |
|
||||
|
||||
## The spin sweep (RFC 004)
|
||||
|
||||
Spinning workers trade idle CPU for wake latency: an idle worker polls
|
||||
`queue_len` for up to `spin_budget_cycles` (at most `max_spinners` at once)
|
||||
before falling back to a futex park, so work that lands during the spin is
|
||||
claimed with no syscall. `spin_budget_cycles = 0` is the feature fully off —
|
||||
the historical `thread::sleep` park, exactly preserved. `spin_sweep` measures
|
||||
both sides of that trade across the budget so the curve can pick the default.
|
||||
|
||||
```
|
||||
cargo bench --bench spin_sweep
|
||||
# on a big box (scale the work up from the sub-second sandbox defaults):
|
||||
SMARM_BENCH_THREADS="1 2 4 8 16" SMARM_BENCH_RUNS=9 \
|
||||
SMARM_SPIN_BUDGET="0 1000 5000 20000 100000 500000 2000000" \
|
||||
SMARM_SPIN_ROUNDS=2000 SMARM_SPIN_BURSTS=500 SMARM_SPIN_ITEMS=5000 \
|
||||
cargo bench --bench spin_sweep | tee bench_results/spin.txt
|
||||
grep '^SPINCSV' bench_results/spin.txt > bench_results/spin.csv
|
||||
```
|
||||
|
||||
Two axes per config: **latency** (p50/p90/p99/min/max, pooled across runs) and
|
||||
**cost** (cores-busy = process CPU-seconds via `getrusage(RUSAGE_SELF)` over
|
||||
wall-seconds; `1.0` = one core pegged, up to `max_spinners` when every spinner
|
||||
is hot). The four workloads:
|
||||
|
||||
- **remote-wake** — the latency target. A consumer parks on `recv`; a producer
|
||||
on another scheduler timestamps just before `send`. Latency is send→receipt.
|
||||
- **fork-join** — fan-out wake latency: root forks `FANOUT` actors, joins them,
|
||||
gap, repeat.
|
||||
- **half-load** — the cost target. Work dispatched every `PERIOD_US`, so workers
|
||||
idle a fraction of each period; cores-busy climbs with budget as spin fills
|
||||
the gaps. Spin cost scales with **park frequency** × budget, so this (not
|
||||
pure-idle) is where the cost curve lives.
|
||||
- **pure-idle** — a floor check: one actor sleeps, nothing else runs. Confirms
|
||||
budget 0 stays parked and a large budget doesn't leak *continuous* CPU (a
|
||||
spinner spends its budget once per park, so a truly idle window barely moves).
|
||||
|
||||
The crossover the sweep reveals: spinning only wins when work arrives while a
|
||||
worker is still inside its budget. With a fixed inter-arrival gap, budgets whose
|
||||
spin-time (`budget_cycles / TSC_freq`) exceeds the gap start catching work
|
||||
before the park — so `SMARM_SPIN_GAP_US` sets the knee; tune it against your
|
||||
box's TSC frequency.
|
||||
|
||||
### Knobs (env vars, all optional)
|
||||
|
||||
| var | default | used by |
|
||||
|---|---|---|
|
||||
| `SMARM_BENCH_THREADS` | `"1 2"` | scheduler-count sweep |
|
||||
| `SMARM_BENCH_RUNS` | `3` | runs per config (latency pooled, cores median) |
|
||||
| `SMARM_SPIN_BUDGET` | `"0 1000 10000 100000 1000000"` | `spin_budget_cycles` sweep |
|
||||
| `SMARM_MAX_SPINNERS` | `"-"` | `max_spinners` sweep; `-` = runtime default (N/2) |
|
||||
| `SMARM_SPIN_GAP_US` | `150` | idle gap between rounds/bursts (sets the latency knee) |
|
||||
| `SMARM_SPIN_ROUNDS` | `100` | remote-wake rounds |
|
||||
| `SMARM_SPIN_FANOUT` / `_BURSTS` | `64` / `30` | fork-join |
|
||||
| `SMARM_SPIN_ITEMS` / `_PERIOD_US` | `150` / `300` | half-load |
|
||||
| `SMARM_SPIN_WINDOW_MS` | `25` | pure-idle window |
|
||||
|
||||
Output is the house table plus a greppable `SPINCSV,...` line per config.
|
||||
|
||||
**On 1 core spinning is pathological** — a hot spinner starves the very thread
|
||||
that would enqueue work, so the optimal budget there is 0 and you only ever see
|
||||
the cost, never the benefit (the sweep will show remote-wake latency getting
|
||||
*worse* with budget on one core). One core validates the harness; the real curve
|
||||
comes from the many-core box.
|
||||
|
||||
## Reading the numbers honestly
|
||||
|
||||
- **Core count is the experiment.** On a 1-core machine (CI, sandboxes) the
|
||||
|
||||
@@ -243,6 +243,90 @@ def check_regressions(current: dict, baseline: dict) -> bool:
|
||||
# Pretty print
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _threads(label: str) -> int | None:
|
||||
"""Worker-thread count implied by a runtime label.
|
||||
|
||||
tokio's `current_thread` is a single-threaded executor (1); an explicit
|
||||
`multi N-thread` is N; a bare `multi-thread` has no count (None) — tokio's
|
||||
default work-stealing pool, paired against smarm's widest config.
|
||||
"""
|
||||
if "current_thread" in label:
|
||||
return 1
|
||||
m = re.search(r"(\d+)-thread", label)
|
||||
return int(m.group(1)) if m else None
|
||||
|
||||
|
||||
def vs_tokio(results: dict) -> list[tuple]:
|
||||
"""Per bench, like-for-like smarm-vs-tokio rows matched by thread count.
|
||||
|
||||
Exact thread-count matches are paired directly (smarm 1-thread vs tokio
|
||||
current_thread, smarm 4-thread vs tokio multi 4-thread, …). tokio's bare
|
||||
`multi-thread` (no explicit count) is paired against the widest unmatched
|
||||
smarm multi-thread config. Lower median µs = faster; ratio = tokio_med /
|
||||
smarm_med, so ratio > 1 means smarm is that many times faster.
|
||||
|
||||
Returns rows of (bench, smarm_label, smarm_med, tokio_label, tokio_med,
|
||||
ratio, winner). Benches without a comparable pair are skipped.
|
||||
"""
|
||||
rows: list[tuple] = []
|
||||
for bench, runtimes in sorted(results.items()):
|
||||
smarm: dict[int, tuple[str, int]] = {}
|
||||
tokio: dict[int, tuple[str, int]] = {}
|
||||
tokio_default: tuple[str, int] | None = None # bare 'multi-thread'
|
||||
for label, data in runtimes.items():
|
||||
n = _threads(label)
|
||||
if label.startswith("smarm"):
|
||||
if n is not None:
|
||||
smarm[n] = (label, data["median"])
|
||||
elif label.startswith("tokio"):
|
||||
if n is None:
|
||||
tokio_default = (label, data["median"])
|
||||
else:
|
||||
tokio[n] = (label, data["median"])
|
||||
|
||||
def row(s: tuple[str, int], t: tuple[str, int]):
|
||||
s_label, s_med = s
|
||||
t_label, t_med = t
|
||||
if s_med == 0:
|
||||
return None
|
||||
ratio = t_med / s_med
|
||||
return (bench, s_label, s_med, t_label, t_med, ratio,
|
||||
"smarm" if ratio >= 1.0 else "tokio")
|
||||
|
||||
matched: set[int] = set()
|
||||
for n in sorted(set(smarm) & set(tokio)):
|
||||
r = row(smarm[n], tokio[n])
|
||||
if r:
|
||||
rows.append(r)
|
||||
matched.add(n)
|
||||
# tokio's default multi pool vs the widest smarm config not already paired.
|
||||
if tokio_default is not None:
|
||||
rem = [n for n in smarm if n not in matched and n > 1]
|
||||
if rem:
|
||||
r = row(smarm[max(rem)], tokio_default)
|
||||
if r:
|
||||
rows.append(r)
|
||||
return rows
|
||||
|
||||
|
||||
def print_vs_tokio(results: dict) -> None:
|
||||
"""Human summary + greppable VSTOKIO lines (best smarm vs best tokio)."""
|
||||
rows = vs_tokio(results)
|
||||
if not rows:
|
||||
return
|
||||
print("\n vs tokio (like-for-like by thread count; ratio>1 = smarm faster, lower µs better)")
|
||||
print(f" {'-'*78}")
|
||||
for bench, s_label, s_med, t_label, t_med, ratio, winner in rows:
|
||||
print(
|
||||
f" {bench:<22} {s_label} {s_med}µs vs {t_label} {t_med}µs"
|
||||
f" → {ratio:.2f}x ({winner})"
|
||||
)
|
||||
# Machine-readable, one line per bench:
|
||||
# VSTOKIO,<bench>,<smarm_label>,<smarm_us>,<tokio_label>,<tokio_us>,<ratio>,<winner>
|
||||
for bench, s_label, s_med, t_label, t_med, ratio, winner in rows:
|
||||
print(f"VSTOKIO,{bench},{s_label},{s_med},{t_label},{t_med},{ratio:.3f},{winner}")
|
||||
|
||||
|
||||
def print_results(results: dict, label: str = "") -> None:
|
||||
if label:
|
||||
print(f"\n{'='*70}")
|
||||
@@ -257,6 +341,7 @@ def print_results(results: dict, label: str = "") -> None:
|
||||
f" {rt_label:>28} | {data['result']:>10} | "
|
||||
f"{data['median']:>10} | {data['min']:>8} | {data['max']:>8}"
|
||||
)
|
||||
print_vs_tokio(results)
|
||||
|
||||
|
||||
def print_sweep_table(sweep_results: list[tuple[int, int, dict]]) -> None:
|
||||
|
||||
Reference in New Issue
Block a user