smarm/benches

Benches

Two families live here: comparison benches (smarm vs tokio, predating v0.5) and the run-queue shootout (v0.5 phase 4). All are plain binaries (harness = false in Cargo.toml), so cargo bench just builds in release and runs main() — no criterion, no magic.

cargo bench --bench <name>           # one bench
cargo bench                          # all of them (slow; rarely what you want)

Catalog

file	what it measures
primes.rs	Compute fan-out/fan-in: counts primes across W workers. Pure compute throughput + spawn/join/channel cost.
multi_scheduler.rs	The original cross-runtime matrix: smarm (1 thread / N threads) vs tokio (current_thread / multi_thread) on compute, ping-pong, and spawn throughput.
general.rs	Workloads where neither runtime has a structural edge. Large gaps here mean real per-task/per-yield overhead differences — watch these for regressions.
smarm_favored.rs	Workloads the stackful green-thread model is built for. Single-thread numbers isolate per-switch cost from contention.
tokio_favored.rs	Workloads tokio's model is built for. Expect to lose; the value is knowing by how much and catching the gap widening.
rq_micro.rs	Run-queue structures in isolation (no runtime, no actors): push/pop throughput sweeping thread count × producer:consumer ratio. Covers all three queue types in one binary — the types compile in every build; only the runtime's alias is feature-selected.
rq_runtime.rs	The whole scheduler with the compile-time-selected queue: yield-storm (pure queue churn), ping-pong-pairs (park/unpark latency), spawn-storm (slab + free list + queue churn), sweeping scheduler count. Comparing variants requires rebuilding per rq-* feature.
spin_sweep.rs	RFC 004 spinning workers: wake latency vs idle-CPU over spin_budget_cycles. Four workloads (remote-wake, fork-join, half-load, pure-idle) × budget × max_spinners × scheduler count; latency percentiles plus cores-busy from getrusage. The data that picks the budget default and on/off posture.

The run-queue shootout

One command; it rebuilds rq_runtime once per queue variant, runs rq_micro once, and aggregates:

./scripts/bench_rq.sh
# on a big box:
SMARM_BENCH_THREADS="1 2 4 8 16 20" ./scripts/bench_rq.sh

Outputs land in bench_results/ (gitignored): one full log per run, plus summary.csv assembled from the machine-readable RQCSV,... lines every config prints alongside the human table.

Manual single-variant runs need the feature dance (features are additive, so the default rq-mutex must be switched off):

cargo bench --bench rq_runtime --no-default-features --features rq-striped

Knobs (env vars, all optional)

var	default	used by
SMARM_BENCH_THREADS	"1 2 4"	both — space-separated sweep
SMARM_BENCH_RUNS	5	both — repetitions; the median is reported
SMARM_BENCH_ITEMS	200000	rq_micro — items per measurement
SMARM_BENCH_YIELD_ACTORS / _YIELDS	200 / 500	rq_runtime yield-storm
SMARM_BENCH_PAIRS / _ROUNDTRIPS	32 / 1000	rq_runtime ping-pong
SMARM_BENCH_SPAWNS	5000	rq_runtime spawn-storm

The spin sweep (RFC 004)

Spinning workers trade idle CPU for wake latency: an idle worker polls queue_len for up to spin_budget_cycles (at most max_spinners at once) before falling back to a futex park, so work that lands during the spin is claimed with no syscall. spin_budget_cycles = 0 is the feature fully off — the historical thread::sleep park, exactly preserved. spin_sweep measures both sides of that trade across the budget so the curve can pick the default.

cargo bench --bench spin_sweep
# on a big box (scale the work up from the sub-second sandbox defaults):
SMARM_BENCH_THREADS="1 2 4 8 16" SMARM_BENCH_RUNS=9 \
SMARM_SPIN_BUDGET="0 1000 5000 20000 100000 500000 2000000" \
SMARM_SPIN_ROUNDS=2000 SMARM_SPIN_BURSTS=500 SMARM_SPIN_ITEMS=5000 \
  cargo bench --bench spin_sweep | tee bench_results/spin.txt
grep '^SPINCSV' bench_results/spin.txt > bench_results/spin.csv

Two axes per config: latency (p50/p90/p99/min/max, pooled across runs) and cost (cores-busy = process CPU-seconds via getrusage(RUSAGE_SELF) over wall-seconds; 1.0 = one core pegged, up to max_spinners when every spinner is hot). The four workloads:

• remote-wake — the latency target. A consumer parks on recv; a producer on another scheduler timestamps just before send. Latency is send→receipt.
• fork-join — fan-out wake latency: root forks FANOUT actors, joins them, gap, repeat.
• half-load — the cost target. Work dispatched every PERIOD_US, so workers idle a fraction of each period; cores-busy climbs with budget as spin fills the gaps. Spin cost scales with park frequency × budget, so this (not pure-idle) is where the cost curve lives.
• pure-idle — a floor check: one actor sleeps, nothing else runs. Confirms budget 0 stays parked and a large budget doesn't leak continuous CPU (a spinner spends its budget once per park, so a truly idle window barely moves).

The crossover the sweep reveals: spinning only wins when work arrives while a worker is still inside its budget. With a fixed inter-arrival gap, budgets whose spin-time (budget_cycles / TSC_freq) exceeds the gap start catching work before the park — so SMARM_SPIN_GAP_US sets the knee; tune it against your box's TSC frequency.

Knobs (env vars, all optional)

var	default	used by
SMARM_BENCH_THREADS	"1 2"	scheduler-count sweep
SMARM_BENCH_RUNS	3	runs per config (latency pooled, cores median)
SMARM_SPIN_BUDGET	"0 1000 10000 100000 1000000"	spin_budget_cycles sweep
SMARM_MAX_SPINNERS	"-"	max_spinners sweep; - = runtime default (N/2)
SMARM_SPIN_GAP_US	150	idle gap between rounds/bursts (sets the latency knee)
SMARM_SPIN_ROUNDS	100	remote-wake rounds
SMARM_SPIN_FANOUT / _BURSTS	64 / 30	fork-join
SMARM_SPIN_ITEMS / _PERIOD_US	150 / 300	half-load
SMARM_SPIN_WINDOW_MS	25	pure-idle window

Output is the house table plus a greppable SPINCSV,... line per config.

On 1 core spinning is pathological — a hot spinner starves the very thread that would enqueue work, so the optimal budget there is 0 and you only ever see the cost, never the benefit (the sweep will show remote-wake latency getting worse with budget on one core). One core validates the harness; the real curve comes from the many-core box.

Reading the numbers honestly

• Core count is the experiment. On a 1-core machine (CI, sandboxes) the sweep only validates the harness and catches gross pathologies — oversubscribed schedulers measure context-switch noise, not contention. Variant decisions come from a many-core box.
• The striped queue should lose at low thread counts (ticket overhead with no contention to amortize) — that's expected, not a bug.
• Medians over SMARM_BENCH_RUNS absorb scheduling noise but not thermal / turbo drift; for publishable numbers, pin the CPU governor and run a warmup pass first.
• spawn-storm batches joins (1024 at a time) to stay well under the slab cap; if you raise SMARM_BENCH_SPAWNS massively, that batching is why it still works.

Adding a bench

1. benches/<name>.rs with a plain main(); print the house table (see any existing bench) and, if it belongs to a sweep, a greppable CSV line with a distinctive prefix (RQCSV, for the shootout family).
2. Register it in Cargo.toml:

   [[bench]]
   name = "<name>"
   harness = false
   

3. Take parameters from SMARM_BENCH_* env vars with modest defaults — the defaults must finish in seconds on one core, the env scales them up on real hardware.
4. Report medians, and keep one measurement = one fresh runtime (init(Config::exact(t)) inside the measured closure constructor, the run() inside the timed region) so runs don't contaminate each other.