perf(preempt): rdtsc unserialised by default; causal attribution opts into lfence
reset_timeslice paid an lfence pipeline drain on every resume via the shared rdtsc() helper. Nothing in preempt.rs needs it: the timeslice arm/expiry compare against a ~1e5-cycle slice, and an early stamp only makes the slice look more used. The consumer that does need it — causal site attribution, where a speculative early read misattributes a site's tail — now calls rdtsc_serialising() explicitly (cold_check sample, SiteGuard enter/exit). Measured vs 31dc26a (baseline commit), 24-core box, rq-mpmc default: - switch_cost, taskset -c 2, 9 interleaved old/new pairs: mean_cyc 149 -> 117 per roundtrip (-32, -21%); mean_ns 43.2 -> 34.9. - sweep.py regress + run, 20T, two runs agree: yield_in_hot_loop 1T 40171 -> 30277/30497 us (-24%) yield_many 1T 12513 -> 9965/10013 us (-20%) ping_pong_oneshot unchanged (RFC 005 wake-slot handoffs skip reset_timeslice, so that path never paid the lfence). All other rows within the box's ~+-15% noise floor. - 1-core sandbox: 218 -> 206 cyc (understates by ~3x). - bench binary lfence count 24 -> 4 (survivors = the bench's own rdtscp;lfence bracket). Tests green. Baseline (benches/baseline.json) not re-saved in this commit.
This commit is contained in:
+8
-3
@@ -279,7 +279,9 @@ mod inner {
|
||||
if slot.is_null() {
|
||||
return;
|
||||
}
|
||||
let now = preempt::rdtsc();
|
||||
// Serialised: `now` closes an interval attributed to a code site; a
|
||||
// speculative early read would drop that site's tail (RFC 007).
|
||||
let now = preempt::rdtsc_serialising();
|
||||
let last = LAST_SAMPLE_TSC.with(|c| c.replace(now));
|
||||
let target_site = (exp >> 32) as u32;
|
||||
let pct = exp & 0xffff_ffff;
|
||||
@@ -374,7 +376,8 @@ mod inner {
|
||||
let target = (exp >> 32) as u32;
|
||||
let pct = exp & 0xffff_ffff;
|
||||
if old == target && new != target {
|
||||
let now = preempt::rdtsc();
|
||||
// Serialised: guard exit bounds the site's interval exactly.
|
||||
let now = preempt::rdtsc_serialising();
|
||||
let last = LAST_SAMPLE_TSC.with(|c| c.replace(now));
|
||||
if pct > 0 {
|
||||
if last != 0 {
|
||||
@@ -386,7 +389,9 @@ mod inner {
|
||||
}
|
||||
}
|
||||
} else if new == target && old != target {
|
||||
LAST_SAMPLE_TSC.with(|c| c.set(preempt::rdtsc()));
|
||||
// Serialised: an early arm would let pre-site work leak into the
|
||||
// first in-site interval — the over-attribution this guards.
|
||||
LAST_SAMPLE_TSC.with(|c| c.set(preempt::rdtsc_serialising()));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user