From 3e321eaaf3667b8df815c1ac5ccb61ea921d5336 Mon Sep 17 00:00:00 2001 From: smarm-agent Date: Sat, 20 Jun 2026 18:51:00 +0000 Subject: [PATCH] pg: rewrite docs for users; relocate internals to items Reframe the module doc in the gen_server house style: lead with what a process group is and when to reach for one, the auto-eviction-on-death behaviour, groups-vs-registry, and a running-context note. Keep the runnable example; add an ignored dispatch/worker-pool example. Move implementation reasoning to where a maintainer stands: lock discipline onto the ProcessGroups store, the eager-cleanup rationale onto reap_group, the join finalize-race detail into join's body comment. Drop all RFC references and the stray phase marker, and lead the public read/select fn docs with what the caller gets. Demote the pub(crate) assert_type intra-doc link in pick_as to plain code, clearing a pre-existing broken-link warning. --- src/pg.rs | 218 ++++++++++++++++++++++++++++++++---------------------- 1 file changed, 128 insertions(+), 90 deletions(-) diff --git a/src/pg.rs b/src/pg.rs index 8e8cc40..7581b42 100644 --- a/src/pg.rs +++ b/src/pg.rs @@ -1,15 +1,17 @@ -//! Process groups — a `name → multiset` map (RFC 012). +//! Process groups: one name, many actors. //! -//! Sits parallel to [`registry`](crate::registry), not on top of it. The -//! registry is a *bimap*: at most one pid per name. A group is the opposite — -//! many pids per name, the same pid in many groups — so it cannot be a -//! generalization of the bimap; it is its own keyspace sharing only the -//! liveness signal source (monitors) and the lock *class*. +//! A process group is a named set of actors that you can look up, fan out to, +//! or pick a worker from. It is the natural home for a *worker pool* (several +//! interchangeable actors doing the same job), for *service discovery* (find +//! everyone currently offering some capability), and for *broadcast* (reach +//! every member of a group at once). //! -//! ## Example +//! The set is *live*: members [`join`] it, and a member that dies is removed +//! automatically. You never deregister a dead actor — there is no bookkeeping +//! to get wrong, and [`members`] / [`pick`] never hand you an actor that has +//! already gone. //! -//! A group is a live membership view: members join, and a member that dies is -//! evicted automatically — no deregistration call, no bookkeeping. +//! ## Joining and reading a group //! //! ``` //! use smarm::{channel, join, leave, members, pick, run, spawn}; @@ -25,8 +27,8 @@ //! join("pool", w2.pid()); //! assert_eq!(members("pool").len(), 2); //! -//! // One worker dies. Nobody told the group — the death hook evicts it, -//! // so it is gone from `members` and never returned by `pick`. +//! // One worker dies. Nothing tells the group — smarm evicts it +//! // automatically, so it is gone from `members` and never picked. //! tx1.send(()).unwrap(); //! w1.join().unwrap(); //! assert_eq!(members("pool"), vec![w2.pid()]); @@ -41,60 +43,72 @@ //! }); //! ``` //! -//! ## Cleanup is eager and monitor-driven (unlike the registry) +//! [`members`] returns every live member in the order they joined; [`pick`] +//! returns one of them, or `None` when the group is empty. The same actor can +//! belong to any number of groups at once, and joining a group it is already in +//! is a harmless no-op. //! -//! The registry prunes a stale binding lazily, on contact, because it only -//! ever resolves one binding at a time. A group is *iterated* — `members` fans -//! out to every member — so it must not carry dead members across a broadcast. -//! Each [`join`] installs a `monitor(pid)` and keeps the resulting [`Monitor`] -//! *alongside the group entry*; the actor's one-shot `Down` lands in that -//! monitor's channel when it dies. Every group operation [`reap`s][reap] the -//! group it touches first — draining each membership's monitor with a -//! non-blocking `try_recv` — and on the first sign of death sweeps that pid out -//! of *every* group via the predicate primitive. So the set is self-pruning on -//! contact rather than filtered on read; the read path additionally applies a -//! generation-checked liveness backstop so a member already dead but not yet -//! reaped is never *returned*, even though eviction remains the monitor's job. +//! ## Membership ends on its own //! -//! [reap]: ProcessGroups::reap_group +//! You do not have to clean up after a member that dies. When an actor exits — +//! for any reason — it is removed from every group it had joined, before any +//! later read or send can observe it. [`leave`] is only for *voluntary* +//! departure, when a still-living actor wants out of a group. //! -//! ## Eviction is one dumb primitive +//! This is the main difference from keeping your own `Vec`: a plain list +//! goes stale the instant a member dies, and you would have to notice and prune +//! it yourself. A group prunes itself. //! -//! [`ProcessGroups::remove_where`] removes every member matching a predicate -//! from every group. The primitive never knows *why* a member leaves — that is -//! the caller's concern. Its first caller is the monitor death hook (this RFC, -//! via `reap_group`); the later `evict_incarnation(node, inc)` sweep (RFC 010) -//! reuses the same predicate path, which is the whole reason to shape it as a -//! predicate. +//! ## Sending to a group //! -//! ## Identity is cluster-shaped from the first commit +//! For a worker pool you usually want to hand a job to *one* available member. +//! [`dispatch`] picks a live member and sends it a message in a single step, +//! returning the member it reached: //! -//! A [`Member`] is not a bare [`Pid`]: it is `(NodeId, Incarnation, Pid)`, a -//! field-for-field image of a modern BEAM pid (`NEW_PID_EXT`) so the eventual -//! ETF codec (RFC 011) is a near-identity mapping. In this RFC `NodeId` and -//! `Incarnation` are runtime-init constants — one node, one fixed incarnation -//! — threaded through storage and eviction anyway so the public surface never -//! has to change to acquire them once clustering (RFC 010) supplies real -//! values. No cluster types leak out of the public functions: callers pass and -//! receive [`Pid`]; the node/incarnation are filled from runtime identity. +//! ```ignore +//! use smarm::{dispatch, join, Addressable}; //! -//! ## Locking +//! struct Job(String); +//! struct Worker; +//! impl Addressable for Worker { type Msg = Job; } //! -//! One `RawMutex` (Leaf class) on `RuntimeInner`, mirroring `registry`. The -//! group lock is never held with another Leaf (it never touches the registry -//! or a slot's cold lock). The two places that *do* need another lock are kept -//! off the group-lock path: +//! // Each worker has published a `Pid` inbox and joined the pool. +//! join("workers", worker_a); +//! join("workers", worker_b); //! -//! - `monitor()` / `demonitor()` acquire the target's cold lock (Leaf) and -//! so run *before* / *after* the group lock, never under it. -//! - draining a monitor with `try_recv` takes the channel's Channel-class -//! lock — permitted *under* a Leaf by the lock order (`raw_mutex.rs`), and -//! a channel critical section only does the lock-free unpark protocol, so -//! no Leaf is ever nested under it. +//! // Route one job to whichever live worker `pick` lands on. +//! match dispatch::("workers", Job("resize image".into())) { +//! Ok(who) => println!("sent to {who:?}"), +//! Err(returned) => println!("no worker took it: {returned:?}"), +//! } +//! ``` //! -//! Evicted and rejected [`Monitor`]s are dropped only *after* the group lock is -//! released, so a receiver-drop never runs a wakeup under the lock (same -//! discipline as `demonitor`). +//! When you want the pids themselves rather than to send right away, [`pick_as`] +//! and [`members_as`] return typed [`Pid`](Pid)s for a homogeneous group, so +//! the follow-up send stays compile-checked. The untyped [`pick`] and +//! [`members`] are for mixed groups, where all you can rely on is identity. +//! +//! ## Groups vs. the registry +//! +//! A group is the many-actors counterpart to the [`registry`](crate::registry). +//! The registry binds a name to *at most one* actor and re-resolves it on every +//! send — what you want for a single well-known service. A group binds a name to +//! *many* actors, and one actor may sit in many groups. Reach for the registry +//! when there is exactly one of something; reach for a group when there is a set. +//! +//! ## Identity and clustering +//! +//! A group member is described by a [`Member`] — a [`Pid`] plus a [`NodeId`] and +//! an [`Incarnation`]. Today everything is single-node, those two fields are +//! fixed defaults, and you only ever pass and receive a plain [`Pid`]: the extra +//! identity is carried so this API will not have to change when groups learn to +//! span a cluster. +//! +//! ## Running context +//! +//! Every function here addresses the current runtime, so each must be called +//! from inside [`run`](crate::run) (that is, on an actor thread). Calling one +//! from outside a running runtime panics. use crate::monitor::{demonitor, monitor, Monitor}; use crate::pid::{assert_type, Addressable, Pid}; @@ -103,7 +117,7 @@ use crate::scheduler::with_runtime; use std::collections::HashMap; /// A cluster node handle. A `u32` integer handle, *not* an interned atom — the -/// single deliberate divergence from the BEAM wire shape (RFC 011 names it). +/// single deliberate divergence from the BEAM wire shape. #[derive(Copy, Clone, PartialEq, Eq, Hash, Debug)] pub struct NodeId(u32); @@ -149,9 +163,8 @@ impl From for Incarnation { } } -/// The fixed single-node identity used until clustering (RFC 010) supplies -/// real values. Carried like `wake_slot` so the public API never has to change -/// to acquire it. +/// The fixed single-node identity used until clustering supplies real values. +/// Carried like `wake_slot` so the public API never has to change to acquire it. pub const DEFAULT_NODE_ID: NodeId = NodeId(0); /// The fixed incarnation for the single-node default. Non-zero so it never /// collides with a BEAM "any creation" wildcard at interop time. @@ -161,7 +174,7 @@ pub const DEFAULT_INCARNATION: Incarnation = Incarnation(1); /// /// Deliberately a field-for-field image of a modern BEAM pid (`NEW_PID_EXT`). /// In memory it is a plain struct — no wire packing; the packed representation -/// belongs to the `RemoteRef` boundary (RFC 010/011), not here. +/// belongs to the remote-reference boundary, not here. #[derive(Copy, Clone, PartialEq, Eq, Hash, Debug)] pub struct Member { /// Which node the pid lives on. `DEFAULT_NODE_ID` while single-node. @@ -174,7 +187,7 @@ pub struct Member { } /// One membership: a [`Member`] and the [`Monitor`] that watches its liveness. -/// The monitor lives *alongside* the group entry (RFC 012) so a group is +/// The monitor lives *alongside* the group entry so a group is /// self-contained: draining the membership tells us whether the member is /// still alive, and dropping the membership drops its monitor. struct Membership { @@ -185,7 +198,23 @@ struct Membership { /// The store: `name → multiset`. Within a single group a `Member` /// appears at most once (`join` is idempotent); the *multiset* framing is for /// cluster-readiness — the same pid is freely a member of many groups, and the -/// width admits multiples in general. Held under one Leaf-class `RawMutex`. +/// width admits multiples in general. +/// +/// Locking discipline. Held under one Leaf-class `RawMutex` on `RuntimeInner`, +/// mirroring the registry, and never held together with another Leaf lock (it +/// never touches the registry or a slot's cold lock). The two operations that +/// do need another lock are kept off the group-lock path: +/// +/// - `monitor()` / `demonitor()` take the target's cold lock (also Leaf), so +/// they run *before* / *after* the group lock, never under it. +/// - draining a monitor with `try_recv` takes the channel's Channel-class +/// lock, which the lock order permits *under* a Leaf; a channel critical +/// section only does the lock-free unpark protocol, so no Leaf ever nests +/// under it. +/// +/// Evicted and rejected [`Monitor`]s are therefore dropped only *after* the +/// group lock is released, so a receiver-drop never runs a wakeup under the +/// lock — the same discipline as `demonitor`. pub(crate) struct ProcessGroups { groups: HashMap>, } @@ -223,9 +252,11 @@ impl ProcessGroups { /// The one dumb eviction primitive: drop every member matching `pred` from /// every group, pruning emptied groups, and return the evicted memberships' /// monitors for the caller to drop outside the lock. The primitive does not - /// know *why* a member leaves. Callers: the death hook (`reap_group`) and, - /// later, `evict_incarnation` (RFC 010), over the same path. Insertion - /// order within a group is preserved (`members` / `pick` are order-stable). + /// know *why* a member leaves; that is the caller's concern. Its callers are + /// the death hook (`reap_group`) and, once clustering lands, an + /// incarnation-eviction sweep — both over this same predicate path, which is + /// the whole reason to shape eviction as a predicate. Insertion order within + /// a group is preserved (`members` / `pick` are order-stable). fn remove_where(&mut self, mut pred: impl FnMut(&Member) -> bool) -> Vec { let mut evicted = Vec::new(); self.groups.retain(|_, v| { @@ -242,12 +273,18 @@ impl ProcessGroups { evicted } - /// Drain-on-contact death hook. Drains every membership monitor in `group` - /// with a non-blocking `try_recv`: a delivered `Down` (any reason) or a - /// closed channel means that member is dead. On the first death detected, - /// sweep *all* of the dead pids out of *every* group via [`remove_where`] - /// — a death is removed from each group it joined, not just the one being - /// touched. Returns the evicted monitors to drop outside the lock. + /// Drain-on-contact death hook. The registry can prune a stale binding + /// lazily, on contact, because it only ever resolves one binding at a time; + /// a group is *iterated* — `members` fans out to everyone — so it must not + /// carry a dead member across a broadcast. Every group operation reaps the + /// group it touches first. + /// + /// Drains every membership monitor in `group` with a non-blocking + /// `try_recv`: a delivered `Down` (any reason) or a closed channel means + /// that member is dead. On the first death detected, sweep *all* of the + /// dead pids out of *every* group via [`remove_where`] — a death is removed + /// from each group it joined, not just the one being touched. Returns the + /// evicted monitors to drop outside the lock. fn reap_group(&mut self, group: &str) -> Vec { let dead: Vec = { let Some(v) = self.groups.get(group) else { @@ -279,7 +316,7 @@ impl ProcessGroups { } /// Live members of `group`, in insertion order. The `is_live` oracle is the - /// read-path backstop (Phase 3): a member whose slot is already dead is + /// read-path backstop: a member whose slot is already dead is /// dropped from the *result* even if its `Down` has not been drained yet. /// Backstop only — the entry stays in storage; eviction is the monitor's /// job (`reap_group`). @@ -314,18 +351,19 @@ fn live(inner: &crate::runtime::RuntimeInner, pid: Pid) -> bool { /// pid is a member at most once (idempotent). Returns `true` if this call newly /// added the membership, `false` if it was already a member. /// -/// Installs a `monitor(pid)` whose one-shot `Down` drives eviction: the -/// registration races `finalize_actor` under the slot's cold lock exactly as -/// every other monitor does, so no death slips between the join and the -/// registration. A redundant (idempotent) join tears its extra monitor back -/// down. +/// Installs a monitor on `pid` so the actor's death evicts it from the group +/// automatically — you never have to remove a dead member yourself. A redundant +/// (idempotent) join tears its extra monitor back down. /// /// Panics if called outside `Runtime::run()`. pub fn join(group: impl Into, pid: Pid) -> bool { let group = group.into(); let pid = pid.erase(); // Install the monitor BEFORE taking the group lock: monitor() acquires the - // target's cold lock (Leaf), and two Leaf locks are never held at once. + // target's cold lock (Leaf), and two Leaf locks are never held at once. The + // registration races `finalize_actor` under that cold lock exactly as every + // other monitor does, so no death can slip between the join and the monitor + // being in place. let mon = monitor(pid); let (rejected, reaped) = with_runtime(|inner| { @@ -372,12 +410,13 @@ pub fn leave(group: &str, pid: Pid) -> bool { } } -/// Fan-out read: every live member of `group`. +/// Every live member of `group`, in the order they joined. Returns an empty +/// vector if the group does not exist or has no live members. /// -/// The touched group is reaped first, so dead members are evicted before the -/// read. The generation-checked liveness read is a belt-and-braces backstop: -/// even in the finalize window where a member is already dead but its `Down` -/// has not yet landed, it is dropped from the result. +/// Dead members are never returned: the group is pruned of anything that has +/// died before the read, and as a backstop a member whose slot is already dead +/// is dropped from the result even in the brief window before its death has +/// been fully processed. /// /// Panics if called outside `Runtime::run()`. pub fn members(group: &str) -> Vec { @@ -391,11 +430,10 @@ pub fn members(group: &str) -> Vec { pids } -/// Pool/discovery read: one live member of `group`, or `None` if empty. -/// Stateless first-live selection: the touched group is reaped first, and the -/// generation-checked liveness backstop skips any member already dead but not -/// yet reaped. Smarter routing is an explicitly later, clustered concern per -/// RFC 010. +/// One live member of `group`, or `None` if the group is empty (or every +/// member has died). Selection is a stateless first-live scan in join order, +/// with the same dead-member backstop as [`members`]; smarter, load-aware +/// routing is a later, clustered concern. /// /// Panics if called outside `Runtime::run()`. pub fn pick(group: &str) -> Option { @@ -409,11 +447,11 @@ pub fn pick(group: &str) -> Option { picked } -/// Typed `pick`: one live member of `group` as a [`Pid`] (RFC 014 §4.4). +/// Typed [`pick`]: one live member of `group` as a [`Pid`](Pid). /// For a homogeneous pool every member is an `A`, so the picked member comes /// back typed and dispatch is an ordinary compile-checked [`send_to`] rather /// than the [`send_dyn`](crate::send_dyn) escape hatch. Re-types via the -/// unchecked [`assert_type`] primitive — a wrong `A` degrades to +/// unchecked `assert_type` primitive — a wrong `A` degrades to /// [`SendError::NoChannel`] on the next send, never a misdelivery. /// /// Panics if called outside `Runtime::run()`.