Lost-wakeup: schedule_loop's phase-1 drain uses drain_lock.try_lock(), and try_lock losers skip the completion drain entirely. Both schedulers park on one shared wake pipe and, until now, drained ALL its bytes right after their idle poll_wake returned — outside the drain lock. A loser could therefore eat the byte announcing a completion the winner had not seen (the winner was already past drain_completions when the epoll thread pushed it), and both threads would park with the completion stranded. Because the bridge eventfd is registered EPOLLONESHOT, the kernel had already disarmed it at epoll_wait, so no later write could re-fire it: the runtime slept until an unrelated timer deadline forced another phase-1 pass. Fix: drain_wake_pipe() moves inside the drain guard, immediately before drain_completions(); the two post-poll drains in the Pop::Idle arms are removed. Producers push their completion before writing the byte, so a byte consumed under the guard always has its completion visible to the drain that follows. An unconsumed byte keeps the (level-triggered) idle poll returning instantly, so a try_lock loser spins briefly until the winner releases — it can no longer sleep through stranded work. Found via smarm_beam's ingress-cap drain barrier flaking under CPU load (5/25 loaded suite runs wedged; mid-wedge stacks showed both schedulers in poll_wake with an FdReady stranded and the eventfd disarmed). Post-fix: 60/60 loaded runs green, tight 5.8-6.8s timing band, no stall tail. Root-cause notes: smarm_beam outputs/flake-rootcause-egress-overload.md.
smarm
SMARM — Smarm, Marks Actor Runtime Machinery. A proof-of-concept green-thread actor runtime for Rust.
Implements the core ideas in Achitecture.md: green-thread actors on a
shared heap, scheduled cooperatively, communicating only by Send messages.
Erlang's isolation model without Erlang's copying GC, Rust's zero-copy
ownership transfers without async's function colouring.
The scheduler is multi-threaded — one OS thread per available CPU, all drawing
from a shared run queue. The single-threaded run() entry point is kept as a
convenience wrapper around runtime::init(Config::exact(1)).run(f).
What's here
| Module | What it does |
|---|---|
stack |
mmap'd growable stack with guard page; SIGSEGV on overflow |
context |
#[naked] x86-64 context-switch shims, callee-saved regs only |
preempt |
Allocator-driven preemption; check!() macro for no-alloc loops |
pid |
(index, generation) PIDs; stale handles are detectable, not silent |
actor |
Trampoline + catch_unwind boundary at the actor entry point |
scheduler |
Run queue, slot table, spawn/join, parking, idle path |
channel |
Unbounded MPSC channel; recv parks the actor; recv_timeout bounds it; select/select_timeout park on many receivers at once (ready-index, priority order) |
mutex |
Mutex<T> with mandatory timeout; FIFO waiters; parks the green thread |
timer |
Min-heap of (deadline, reason); Sleep and WaitTimeout reasons |
io |
block_on_io for blocking work; wait_readable/wait_writable + read/write via epoll |
supervisor |
Signal::Exit/Panic/Stopped funnelled to a parent; OneForOne/OneForAll/RestForOne strategies + restart-intensity cap |
monitor |
monitor(pid) → Monitor { id, target, rx }; one-shot Down via rx; demonitor(&m) tears one registration down; unidirectional death notice |
link |
bidirectional link/unlink; abnormal death propagates (cooperative stop, or an ExitSignal message under trap_exit) |
gen_server |
call/call_timeout (sync request-reply) / cast (async) over one inbox; handle_info over static info arms + handle_down via Watcher-fed monitors, selected ahead of the inbox; ServerRef/ServerBuilder + init/terminate hooks; server-down via channel closure |
registry |
register/whereis/name_of: name ↔ pid bimap; lazy generation-checked cleanup |
Quick taste
use smarm::{run, spawn, channel};
run(|| {
let (tx, rx) = channel::<i64>();
let h = spawn(move || {
for _ in 0..3 {
let v = rx.recv().unwrap();
println!("got {v}");
}
});
for v in 1..=3i64 {
tx.send(v).unwrap();
}
h.join().unwrap();
});
Layout
src/
stack.rs context.rs preempt.rs pid.rs actor.rs
scheduler.rs channel.rs mutex.rs timer.rs io.rs
supervisor.rs monitor.rs link.rs runtime.rs
gen_server.rs lib.rs
tests/
per-module integration tests
benches/
primes.rs fan-out/fan-in compute, vs tokio current_thread
Building and running
Standard Cargo. Requires Rust 1.95 or newer (the #[naked] attribute went stable
in 1.88; we use a few unrelated post-1.88 features). master is x86-64 Linux
only. An experimental, untested aarch64 context-switch backend lives on the
arm-port branch (extracted into a target_arch-gated src/arch/); it has not
been validated on hardware yet. macOS remains on the deferred list because of the
epoll dependency.
cargo test # all tests
cargo test --test mutex # one module
cargo bench # primes benchmark vs tokio
What's not here
See the Defer section of Architecture.md.
join! for handle groups, stack growth via remap,
hierarchical timer wheel, fd-wait timeouts, Signal::Timeout. Each is
mechanism we know how to add; none belongs in this iteration.
Docs
| Document | What it covers |
|---|---|
Architecture.md |
Design intent, runtime model, and deferred work |
smarm - Deep Dive.html |
Generated walkthrough of the system; good starting point |
BENCHMARKS_AND_TUNING.md |
Where smarm wins and loses vs tokio, preemption knob recommendations |
benchmarks.md |
Raw benchmark results, methodology, and tuning experiment log |
Contributing
This is a personal proof-of-concept. There's no PR workflow. If you fork it and do something interesting, just send me an email. If it's nice, I'll upstream the changes.