smarm-agent 0017c5b9a1 fix(runtime): consume wake-pipe bytes only under the drain lock
Lost-wakeup: schedule_loop's phase-1 drain uses drain_lock.try_lock(), and
try_lock losers skip the completion drain entirely. Both schedulers park on
one shared wake pipe and, until now, drained ALL its bytes right after their
idle poll_wake returned — outside the drain lock. A loser could therefore
eat the byte announcing a completion the winner had not seen (the winner was
already past drain_completions when the epoll thread pushed it), and both
threads would park with the completion stranded. Because the bridge eventfd
is registered EPOLLONESHOT, the kernel had already disarmed it at
epoll_wait, so no later write could re-fire it: the runtime slept until an
unrelated timer deadline forced another phase-1 pass.

Fix: drain_wake_pipe() moves inside the drain guard, immediately before
drain_completions(); the two post-poll drains in the Pop::Idle arms are
removed. Producers push their completion before writing the byte, so a byte
consumed under the guard always has its completion visible to the drain that
follows. An unconsumed byte keeps the (level-triggered) idle poll returning
instantly, so a try_lock loser spins briefly until the winner releases —
it can no longer sleep through stranded work.

Found via smarm_beam's ingress-cap drain barrier flaking under CPU load
(5/25 loaded suite runs wedged; mid-wedge stacks showed both schedulers in
poll_wake with an FdReady stranded and the eventfd disarmed). Post-fix:
60/60 loaded runs green, tight 5.8-6.8s timing band, no stall tail.
Root-cause notes: smarm_beam outputs/flake-rootcause-egress-overload.md.
2026-07-11 16:13:46 +00:00
2026-05-26 23:14:46 +02:00

smarm

SMARM — Smarm, Marks Actor Runtime Machinery. A proof-of-concept green-thread actor runtime for Rust.

Implements the core ideas in Achitecture.md: green-thread actors on a shared heap, scheduled cooperatively, communicating only by Send messages. Erlang's isolation model without Erlang's copying GC, Rust's zero-copy ownership transfers without async's function colouring.

The scheduler is multi-threaded — one OS thread per available CPU, all drawing from a shared run queue. The single-threaded run() entry point is kept as a convenience wrapper around runtime::init(Config::exact(1)).run(f).

What's here

Module What it does
stack mmap'd growable stack with guard page; SIGSEGV on overflow
context #[naked] x86-64 context-switch shims, callee-saved regs only
preempt Allocator-driven preemption; check!() macro for no-alloc loops
pid (index, generation) PIDs; stale handles are detectable, not silent
actor Trampoline + catch_unwind boundary at the actor entry point
scheduler Run queue, slot table, spawn/join, parking, idle path
channel Unbounded MPSC channel; recv parks the actor; recv_timeout bounds it; select/select_timeout park on many receivers at once (ready-index, priority order)
mutex Mutex<T> with mandatory timeout; FIFO waiters; parks the green thread
timer Min-heap of (deadline, reason); Sleep and WaitTimeout reasons
io block_on_io for blocking work; wait_readable/wait_writable + read/write via epoll
supervisor Signal::Exit/Panic/Stopped funnelled to a parent; OneForOne/OneForAll/RestForOne strategies + restart-intensity cap
monitor monitor(pid)Monitor { id, target, rx }; one-shot Down via rx; demonitor(&m) tears one registration down; unidirectional death notice
link bidirectional link/unlink; abnormal death propagates (cooperative stop, or an ExitSignal message under trap_exit)
gen_server call/call_timeout (sync request-reply) / cast (async) over one inbox; handle_info over static info arms + handle_down via Watcher-fed monitors, selected ahead of the inbox; ServerRef/ServerBuilder + init/terminate hooks; server-down via channel closure
registry register/whereis/name_of: name ↔ pid bimap; lazy generation-checked cleanup

Quick taste

use smarm::{run, spawn, channel};

run(|| {
    let (tx, rx) = channel::<i64>();
    let h = spawn(move || {
        for _ in 0..3 {
            let v = rx.recv().unwrap();
            println!("got {v}");
        }
    });
    for v in 1..=3i64 {
        tx.send(v).unwrap();
    }
    h.join().unwrap();
});

Layout

src/
  stack.rs context.rs preempt.rs pid.rs actor.rs
  scheduler.rs channel.rs mutex.rs timer.rs io.rs
  supervisor.rs monitor.rs link.rs runtime.rs
  gen_server.rs lib.rs
tests/
  per-module integration tests
benches/
  primes.rs    fan-out/fan-in compute, vs tokio current_thread

Building and running

Standard Cargo. Requires Rust 1.95 or newer (the #[naked] attribute went stable in 1.88; we use a few unrelated post-1.88 features). master is x86-64 Linux only. An experimental, untested aarch64 context-switch backend lives on the arm-port branch (extracted into a target_arch-gated src/arch/); it has not been validated on hardware yet. macOS remains on the deferred list because of the epoll dependency.

cargo test                # all tests
cargo test --test mutex   # one module
cargo bench               # primes benchmark vs tokio

What's not here

See the Defer section of Architecture.md. join! for handle groups, stack growth via remap, hierarchical timer wheel, fd-wait timeouts, Signal::Timeout. Each is mechanism we know how to add; none belongs in this iteration.

Docs

Document What it covers
Architecture.md Design intent, runtime model, and deferred work
smarm - Deep Dive.html Generated walkthrough of the system; good starting point
BENCHMARKS_AND_TUNING.md Where smarm wins and loses vs tokio, preemption knob recommendations
benchmarks.md Raw benchmark results, methodology, and tuning experiment log

Contributing

This is a personal proof-of-concept. There's no PR workflow. If you fork it and do something interesting, just send me an email. If it's nice, I'll upstream the changes.


S
Description
SMARM - Smarm, Marks Actor Runtime Machinery
Readme MIT
998 KiB
Languages
Rust 98%
Python 1.7%
Shell 0.3%