155 lines
8.0 KiB
Markdown
155 lines
8.0 KiB
Markdown
# Benches
|
||
|
||
Two families live here: **comparison benches** (smarm vs tokio, predating
|
||
v0.5) and the **run-queue shootout** (v0.5 phase 4). All are plain binaries
|
||
(`harness = false` in `Cargo.toml`), so `cargo bench` just builds in release
|
||
and runs `main()` — no criterion, no magic.
|
||
|
||
```
|
||
cargo bench --bench <name> # one bench
|
||
cargo bench # all of them (slow; rarely what you want)
|
||
```
|
||
|
||
## Catalog
|
||
|
||
| file | what it measures |
|
||
|---|---|
|
||
| `primes.rs` | Compute fan-out/fan-in: counts primes across W workers. Pure compute throughput + spawn/join/channel cost. |
|
||
| `multi_scheduler.rs` | The original cross-runtime matrix: smarm (1 thread / N threads) vs tokio (current_thread / multi_thread) on compute, ping-pong, and spawn throughput. |
|
||
| `general.rs` | Workloads where neither runtime has a structural edge. Large gaps here mean real per-task/per-yield overhead differences — watch these for regressions. |
|
||
| `smarm_favored.rs` | Workloads the stackful green-thread model is built for. Single-thread numbers isolate per-switch cost from contention. |
|
||
| `tokio_favored.rs` | Workloads tokio's model is built for. Expect to lose; the value is knowing *by how much* and catching the gap widening. |
|
||
| `rq_micro.rs` | Run-queue **structures** in isolation (no runtime, no actors): push/pop throughput sweeping thread count × producer:consumer ratio. Covers all three queue types in one binary — the types compile in every build; only the runtime's alias is feature-selected. |
|
||
| `rq_runtime.rs` | The **whole scheduler** with the compile-time-selected queue: yield-storm (pure queue churn), ping-pong-pairs (park/unpark latency), spawn-storm (slab + free list + queue churn), sweeping scheduler count. Comparing variants requires rebuilding per `rq-*` feature. |
|
||
| `spin_sweep.rs` | RFC 004 spinning workers: wake **latency vs idle-CPU** over `spin_budget_cycles`. Four workloads (remote-wake, fork-join, half-load, pure-idle) × budget × `max_spinners` × scheduler count; latency percentiles plus cores-busy from `getrusage`. The data that picks the budget default and on/off posture. |
|
||
|
||
## The run-queue shootout
|
||
|
||
One command; it rebuilds `rq_runtime` once per queue variant, runs `rq_micro`
|
||
once, and aggregates:
|
||
|
||
```
|
||
./scripts/bench_rq.sh
|
||
# on a big box:
|
||
SMARM_BENCH_THREADS="1 2 4 8 16 20" ./scripts/bench_rq.sh
|
||
```
|
||
|
||
Outputs land in `bench_results/` (gitignored): one full log per run, plus
|
||
`summary.csv` assembled from the machine-readable `RQCSV,...` lines every
|
||
config prints alongside the human table.
|
||
|
||
Manual single-variant runs need the feature dance (features are additive, so
|
||
the default `rq-mutex` must be switched off):
|
||
|
||
```
|
||
cargo bench --bench rq_runtime --no-default-features --features rq-striped
|
||
```
|
||
|
||
### Knobs (env vars, all optional)
|
||
|
||
| var | default | used by |
|
||
|---|---|---|
|
||
| `SMARM_BENCH_THREADS` | `"1 2 4"` | both — space-separated sweep |
|
||
| `SMARM_BENCH_RUNS` | `5` | both — repetitions; the **median** is reported |
|
||
| `SMARM_BENCH_ITEMS` | `200000` | `rq_micro` — items per measurement |
|
||
| `SMARM_BENCH_YIELD_ACTORS` / `_YIELDS` | `200` / `500` | `rq_runtime` yield-storm |
|
||
| `SMARM_BENCH_PAIRS` / `_ROUNDTRIPS` | `32` / `1000` | `rq_runtime` ping-pong |
|
||
| `SMARM_BENCH_SPAWNS` | `5000` | `rq_runtime` spawn-storm |
|
||
|
||
## The spin sweep (RFC 004)
|
||
|
||
Spinning workers trade idle CPU for wake latency: an idle worker polls
|
||
`queue_len` for up to `spin_budget_cycles` (at most `max_spinners` at once)
|
||
before falling back to a futex park, so work that lands during the spin is
|
||
claimed with no syscall. `spin_budget_cycles = 0` is the feature fully off —
|
||
the historical `thread::sleep` park, exactly preserved. `spin_sweep` measures
|
||
both sides of that trade across the budget so the curve can pick the default.
|
||
|
||
```
|
||
cargo bench --bench spin_sweep
|
||
# on a big box (scale the work up from the sub-second sandbox defaults):
|
||
SMARM_BENCH_THREADS="1 2 4 8 16" SMARM_BENCH_RUNS=9 \
|
||
SMARM_SPIN_BUDGET="0 1000 5000 20000 100000 500000 2000000" \
|
||
SMARM_SPIN_ROUNDS=2000 SMARM_SPIN_BURSTS=500 SMARM_SPIN_ITEMS=5000 \
|
||
cargo bench --bench spin_sweep | tee bench_results/spin.txt
|
||
grep '^SPINCSV' bench_results/spin.txt > bench_results/spin.csv
|
||
```
|
||
|
||
Two axes per config: **latency** (p50/p90/p99/min/max, pooled across runs) and
|
||
**cost** (cores-busy = process CPU-seconds via `getrusage(RUSAGE_SELF)` over
|
||
wall-seconds; `1.0` = one core pegged, up to `max_spinners` when every spinner
|
||
is hot). The four workloads:
|
||
|
||
- **remote-wake** — the latency target. A consumer parks on `recv`; a producer
|
||
on another scheduler timestamps just before `send`. Latency is send→receipt.
|
||
- **fork-join** — fan-out wake latency: root forks `FANOUT` actors, joins them,
|
||
gap, repeat.
|
||
- **half-load** — the cost target. Work dispatched every `PERIOD_US`, so workers
|
||
idle a fraction of each period; cores-busy climbs with budget as spin fills
|
||
the gaps. Spin cost scales with **park frequency** × budget, so this (not
|
||
pure-idle) is where the cost curve lives.
|
||
- **pure-idle** — a floor check: one actor sleeps, nothing else runs. Confirms
|
||
budget 0 stays parked and a large budget doesn't leak *continuous* CPU (a
|
||
spinner spends its budget once per park, so a truly idle window barely moves).
|
||
|
||
The crossover the sweep reveals: spinning only wins when work arrives while a
|
||
worker is still inside its budget. With a fixed inter-arrival gap, budgets whose
|
||
spin-time (`budget_cycles / TSC_freq`) exceeds the gap start catching work
|
||
before the park — so `SMARM_SPIN_GAP_US` sets the knee; tune it against your
|
||
box's TSC frequency.
|
||
|
||
### Knobs (env vars, all optional)
|
||
|
||
| var | default | used by |
|
||
|---|---|---|
|
||
| `SMARM_BENCH_THREADS` | `"1 2"` | scheduler-count sweep |
|
||
| `SMARM_BENCH_RUNS` | `3` | runs per config (latency pooled, cores median) |
|
||
| `SMARM_SPIN_BUDGET` | `"0 1000 10000 100000 1000000"` | `spin_budget_cycles` sweep |
|
||
| `SMARM_MAX_SPINNERS` | `"-"` | `max_spinners` sweep; `-` = runtime default (N/2) |
|
||
| `SMARM_SPIN_GAP_US` | `150` | idle gap between rounds/bursts (sets the latency knee) |
|
||
| `SMARM_SPIN_ROUNDS` | `100` | remote-wake rounds |
|
||
| `SMARM_SPIN_FANOUT` / `_BURSTS` | `64` / `30` | fork-join |
|
||
| `SMARM_SPIN_ITEMS` / `_PERIOD_US` | `150` / `300` | half-load |
|
||
| `SMARM_SPIN_WINDOW_MS` | `25` | pure-idle window |
|
||
|
||
Output is the house table plus a greppable `SPINCSV,...` line per config.
|
||
|
||
**On 1 core spinning is pathological** — a hot spinner starves the very thread
|
||
that would enqueue work, so the optimal budget there is 0 and you only ever see
|
||
the cost, never the benefit (the sweep will show remote-wake latency getting
|
||
*worse* with budget on one core). One core validates the harness; the real curve
|
||
comes from the many-core box.
|
||
|
||
## Reading the numbers honestly
|
||
|
||
- **Core count is the experiment.** On a 1-core machine (CI, sandboxes) the
|
||
sweep only validates the harness and catches gross pathologies —
|
||
oversubscribed schedulers measure context-switch noise, not contention.
|
||
Variant decisions come from a many-core box.
|
||
- The striped queue *should lose* at low thread counts (ticket overhead with
|
||
no contention to amortize) — that's expected, not a bug.
|
||
- Medians over `SMARM_BENCH_RUNS` absorb scheduling noise but not thermal /
|
||
turbo drift; for publishable numbers, pin the CPU governor and run a warmup
|
||
pass first.
|
||
- `spawn-storm` batches joins (1024 at a time) to stay well under the slab
|
||
cap; if you raise `SMARM_BENCH_SPAWNS` massively, that batching is why it
|
||
still works.
|
||
|
||
## Adding a bench
|
||
|
||
1. `benches/<name>.rs` with a plain `main()`; print the house table (see any
|
||
existing bench) and, if it belongs to a sweep, a greppable CSV line with a
|
||
distinctive prefix (`RQCSV,` for the shootout family).
|
||
2. Register it in `Cargo.toml`:
|
||
```toml
|
||
[[bench]]
|
||
name = "<name>"
|
||
harness = false
|
||
```
|
||
3. Take parameters from `SMARM_BENCH_*` env vars with modest defaults — the
|
||
defaults must finish in seconds on one core, the env scales them up on
|
||
real hardware.
|
||
4. Report **medians**, and keep one measurement = one fresh runtime
|
||
(`init(Config::exact(t))` inside the measured closure constructor, the
|
||
`run()` inside the timed region) so runs don't contaminate each other.
|