fix(runtime): handle timer firing while actor is still Runnable in sleep()

While running benchmarks, a hang surfaced in timer-only workloads — actors
sleeping with no IO in flight. Tracing it down, the race lives in sleep():
between the call to `timers.insert_sleep` and the subsequent `park_current`
yield, the actor is still in State::Runnable. If the timer fires in that
window, the old code's `if matches!(slot.state, State::Parked)` guard silently
drops the wakeup. The actor then parks normally and never gets re-queued —
it sleeps forever.

The fix mirrors what scheduler::unpark() and the IO FdReady path already do:
when the timer fires and the slot is still Runnable, set `pending_unpark`
instead of re-queuing immediately. The upcoming Park yield sees the flag and
re-queues the actor rather than suspending it, closing the race without any
new synchronisation.

Adds a regression test: 100 actors doing pure timer sleeps across ≥2
scheduler threads. The test asserts both correctness (all actors complete)
and timeliness (wall time < 2×sleep duration), which is enough signal to
catch a stuck actor even on a single-core CI runner.
This commit is contained in:
smarm
2026-05-26 21:58:14 +02:00
parent 72f5d38e5d
commit 7746dca69b
6 changed files with 395 additions and 91 deletions
+64
View File
@@ -421,3 +421,67 @@ fn ping_pong_completes() {
});
assert_eq!(final_val.load(Ordering::SeqCst), ROUNDS);
}
/// Regression test for the multi-scheduler timer-only hang.
///
/// Bug: when the run queue is empty and only timers are pending (no IO
/// outstanding), all N scheduler threads called `poll_wake(wake_fd,
/// Some(timeout))` on the *same* pipe fd. When the timer fired, the one
/// thread that won `drain_lock` consumed the single wake byte and re-queued
/// the actors; the other N-1 threads stayed blocked in `poll()` for the full
/// timeout duration. After actors completed and `all_clear` became true, the
/// stuck threads had to wait out the remainder of their poll timeout before
/// noticing — adding up to one full sleep duration of extra latency.
///
/// Fix: use `thread::sleep(timeout)` when `io_outstanding == 0`, so every
/// scheduler thread independently wakes at the deadline without contending
/// on the wake pipe.
///
/// Signal: with SLEEP_MS=300 and ≥2 scheduler threads, the broken impl
/// takes ≥2×SLEEP_MS (actors sleep + stuck threads drain their poll timeout
/// before seeing all_clear). The fixed impl takes ≈SLEEP_MS + epsilon.
#[test]
fn multi_thread_timer_only_no_pipe_contention() {
const SLEEP_MS: u64 = 300;
const ACTORS: usize = 100;
// Need at least 2 scheduler threads: one wins drain_lock and does work,
// the rest pile into poll_wake and get stuck.
let n = std::thread::available_parallelism()
.map(|n| n.get())
.unwrap_or(1)
.max(2);
let r = smarm::runtime::init(Config::exact(n.min(4)));
let count = Arc::new(AtomicU64::new(0));
let c = count.clone();
let start = std::time::Instant::now();
r.run(move || {
let mut handles = Vec::new();
for _ in 0..ACTORS {
let cc = c.clone();
handles.push(spawn(move || {
// Pure timer sleep: no channels, no IO.
// All scheduler threads will be idle simultaneously while
// these timers are pending — the condition that triggers the bug.
smarm::sleep(Duration::from_millis(SLEEP_MS));
cc.fetch_add(1, Ordering::SeqCst);
}));
}
for h in handles {
h.join().unwrap();
}
});
assert_eq!(count.load(Ordering::SeqCst), ACTORS as u64, "not all actors completed");
let elapsed = start.elapsed();
assert!(
elapsed < Duration::from_millis(SLEEP_MS * 2),
"run() took {elapsed:?} — expected <{}ms; likely stuck threads draining \
their poll_wake timeout after all_clear (bug: scheduler threads poll \
the shared wake pipe instead of sleeping independently for timer-only workloads)",
SLEEP_MS * 2,
);
}