An idle scheduler thread blocks in poll_wake on a snapshot of peek_deadline / io_outstanding. Only the io threads ever write the wake pipe (on completion push); enqueue does not. Mid-flight that is masked — the thread that caused an enqueue is awake and processes it — but at termination the snapshot can go terminally stale, two ways: - Stale io_outstanding (infinite hang): an actor parked in wait_readable is request_stop'ped. Cancellation deregisters the waiter and produces NO completion, so the wake pipe is never written. A sibling blocked on io_outstanding > 0 with no timers pending sits in poll(-1) forever. - Orphaned timer deadline (finite stall): an actor cancelled out of a long sleep leaves its timer entry behind. Clearing the wheel at AllDone doesn't help a sibling that already blocked on that deadline; it sleeps it out in full. In both cases the remaining actors finish on the other scheduler thread, which reaches the live==0 && io_out==0 verdict and returns without waking the blocked one. Runtime::run then stalls or hangs in its worker join. Found via urus's graceful-shutdown tests (~10% flake, one test per variant); confirmed by gdb dumps of hung processes showing run() in JoinHandle::join over a sibling in poll(wake_fd, 59750ms) resp. poll(-1). Fix: Io::wake() writes the wake pipe directly; the AllDone arm calls it after the timer clear. One byte wakes every poller; each re-runs the verdict, independently reaches AllDone, and re-wakes — idempotent. tests/terminal_wake.rs reproduces both variants deterministically: the root busy-spins (no timer entries, occupies one scheduler thread) so the sibling settles into the stale idle wait before the stop is issued. Both hang without the fix and pass in <0.5s with it. Known residual gap, deliberately unfixed: the timers-pending / no-io-subsystem idle branch blocks in thread::sleep and has no wake mechanism at all — the same stall exists for runtimes that never initialize io. Roadmap candidate alongside cross-thread unpark and entry-side check_cancelled in park_current (the lossy-QUEUED-stop bug).
smarm
SMARM — Smarm, Marks Actor Runtime Machinery. A proof-of-concept green-thread actor runtime for Rust.
Implements the core ideas in Achitecture.md: green-thread actors on a
shared heap, scheduled cooperatively, communicating only by Send messages.
Erlang's isolation model without Erlang's copying GC, Rust's zero-copy
ownership transfers without async's function colouring.
The scheduler is multi-threaded — one OS thread per available CPU, all drawing
from a shared run queue. The single-threaded run() entry point is kept as a
convenience wrapper around runtime::init(Config::exact(1)).run(f).
What's here
| Module | What it does |
|---|---|
stack |
mmap'd growable stack with guard page; SIGSEGV on overflow |
context |
#[naked] x86-64 context-switch shims, callee-saved regs only |
preempt |
Allocator-driven preemption; check!() macro for no-alloc loops |
pid |
(index, generation) PIDs; stale handles are detectable, not silent |
actor |
Trampoline + catch_unwind boundary at the actor entry point |
scheduler |
Run queue, slot table, spawn/join, parking, idle path |
channel |
Unbounded MPSC channel; recv parks the actor; recv_timeout bounds it; select/select_timeout park on many receivers at once (ready-index, priority order) |
mutex |
Mutex<T> with mandatory timeout; FIFO waiters; parks the green thread |
timer |
Min-heap of (deadline, reason); Sleep and WaitTimeout reasons |
io |
block_on_io for blocking work; wait_readable/wait_writable + read/write via epoll |
supervisor |
Signal::Exit/Panic/Stopped funnelled to a parent; OneForOne/OneForAll/RestForOne strategies + restart-intensity cap |
monitor |
monitor(pid) → Monitor { id, target, rx }; one-shot Down via rx; demonitor(&m) tears one registration down; unidirectional death notice |
link |
bidirectional link/unlink; abnormal death propagates (cooperative stop, or an ExitSignal message under trap_exit) |
gen_server |
call/call_timeout (sync request-reply) / cast (async) over one inbox; handle_info over static info arms + handle_down via Watcher-fed monitors, selected ahead of the inbox; ServerRef/ServerBuilder + init/terminate hooks; server-down via channel closure |
registry |
register/whereis/name_of: name ↔ pid bimap; lazy generation-checked cleanup |
Quick taste
use smarm::{run, spawn, channel};
run(|| {
let (tx, rx) = channel::<i64>();
let h = spawn(move || {
for _ in 0..3 {
let v = rx.recv().unwrap();
println!("got {v}");
}
});
for v in 1..=3i64 {
tx.send(v).unwrap();
}
h.join().unwrap();
});
Layout
src/
stack.rs context.rs preempt.rs pid.rs actor.rs
scheduler.rs channel.rs mutex.rs timer.rs io.rs
supervisor.rs monitor.rs link.rs runtime.rs
gen_server.rs lib.rs
tests/
per-module integration tests
benches/
primes.rs fan-out/fan-in compute, vs tokio current_thread
Building and running
Standard Cargo. Requires Rust 1.95 or newer (the #[naked] attribute went stable
in 1.88; we use a few unrelated post-1.88 features). master is x86-64 Linux
only. An experimental, untested aarch64 context-switch backend lives on the
arm-port branch (extracted into a target_arch-gated src/arch/); it has not
been validated on hardware yet. macOS remains on the deferred list because of the
epoll dependency.
cargo test # all tests
cargo test --test mutex # one module
cargo bench # primes benchmark vs tokio
What's not here
See the Defer section of Architecture.md.
join! for handle groups, stack growth via remap,
hierarchical timer wheel, fd-wait timeouts, Signal::Timeout. Each is
mechanism we know how to add; none belongs in this iteration.
Docs
| Document | What it covers |
|---|---|
Architecture.md |
Design intent, runtime model, and deferred work |
smarm - Deep Dive.html |
Generated walkthrough of the system; good starting point |
BENCHMARKS_AND_TUNING.md |
Where smarm wins and loses vs tokio, preemption knob recommendations |
benchmarks.md |
Raw benchmark results, methodology, and tuning experiment log |
Contributing
This is a personal proof-of-concept. There's no PR workflow. If you fork it and do something interesting, just send me an email. If it's nice, I'll upstream the changes.