The runner registry

Included from Basic tier

The runner registry is the fleet's durable roster: every enrolled runner identity, the registration tokens that mint them, runner groups with their placement contract, the admin-managed attribute registry, and per-runner and per-group drain state. It lives in the shared database, so every replica sees one fleet and a server restart forgets nothing — a restarted server still recognises its runners, keeps a revocation refused, and keeps honouring in-flight jobs.

Its neighbours: the dispatch plane (opschain docs dispatch) matches queued work against the registry's facts; the fleet queue (opschain docs fleet-queue) turns those facts into per-entry verdicts; the agent plane (opschain docs agent-plane) shares the same registry and token machinery for its own member kind.

Registration tokens

Enrolment is by admin-minted, scoped registration token (opschain admin runner token create, or POST /api/registration_tokens):

Field	Meaning
kind	What the token mints — runner (the default) or agent; a token mints exactly one kind.
group	The runner group enrolment inherits; defaults to default, and minting auto-creates the group row if absent.
labels	Capability labels stamped onto every runner the token enrols.
ephemeral	Marks enrolled runners ephemeral.
max_uses	0 = unlimited within the TTL; 1 is the just-in-time single-use shape.
ttl	Lifetime from mint; default 1 hour.

The token value is returned once, at mint, and never again — listings (GET /api/registration_tokens, opschain admin runner token list) serve metadata only, each row named by its non-redeemable fingerprint (sha256: + the first 16 hex characters of the value's SHA-256) with a derived status: active, expired or exhausted (an expired-and-exhausted token reads expired). A refused registration is deliberately opaque to the caller — unknown, expired and exhausted all answer the same authentication error — so the listing's verdict is how an operator diagnoses it.

Minting a token for a new group name is gated by the licence's runner_groups limit, as is group upsert — both count group names identically. Registration itself is gated by the licence's runner limit, with the re-enrolling key's own row excluded so a known runner reconnecting never counts itself out. See opschain docs entitlements.

Runner identity

The Ed25519 keypair is the identity — the display name is not unique, and enrolling an already-enrolled public key supersedes that key's previous registration (one keypair, one row; the old row and its stored job bindings are removed). Registration captures the token's scope (group, labels, ephemeral), the runner's advertised capabilities (OS, arch, executors — see opschain docs runner-executors), its self-reported host inventory (hostname, addresses, CPU count, versions, PID, state dir, the declared --max-parallel-steps capacity — admin-surface data only, never logged), and the server-observed peer address. The host inventory refreshes on session open and poll piggyback.

The inventory carries the runner binary's build identity (version and commit, stamped at build time), and the fleet view serves it with a version_skew verdict against the server's own build — none, skewed or unknown (a runner that reports no identity). A skewed or unknown runner also records one customer-visible warn:runner:version_skew event per (runner, version-pair) at registration and session establishment — never per renewal — so a stale fleet surfaces itself instead of masquerading as a fix "not working".

Three ways out, deliberately distinct:

• Goodbye — the runner's graceful-exit announcement records a stopped marker, so the fleet view reads it stopped immediately; the identity's next poll clears it.
• Forget (POST /api/runners/{id}/forget, opschain admin runner forget) — deletes a stale registration so the list is clean and the identity may re-register later; no tombstone. A runner seen within the liveness window is refused: runner is online: stop or drain it before forgetting it; one holding a live job binding is refused: runner holds a live job binding: drain it and let the job finish before forgetting it.
• Revocation (deletion through the registry) — writes a durable tombstone: the id answers as revoked, not unknown, and the key is refused if it tries to re-register. Revoked ids answer 404 on the admin surfaces, like unknown ones — the tombstone stays un-exposed.

Liveness

Every acquire poll touches the runner's last_seen_at. The server-side state vocabulary, as GET /api/runners reports it:

Field	Meaning
online	Not stopped, and seen within the last 3 minutes (the liveness window).
last_seen_at	The last poll.
stopped_at	The goodbye marker; cleared by the next poll.
draining	A runner-level or group-level drain is in force.
drain_requested_at	When the runner-level drain was requested.
group_draining	The group-level half, separately visible.
current_jobs	Live job bindings (run, step, attempt, acked), enriched with the run's action and target path.
group_attributes / group_exclusive	The group's placement posture.

The detail view (GET /api/runners/{id}, opschain admin runner show) adds registered_at, the observed peer address, the host inventory, the group's bindings and the public key's fingerprint (SHA256:… — never the raw key).

Offline means stopped, or silent past the 3-minute window. Silent disappearance is made loud: the scheduler leader sweeps every minute for registered runners whose last contact aged past the window without a goodbye and emits warn:runner:aged_out — once per offline episode, re-armed when the runner returns (the throttle is checked against the events table, so it holds across replicas).

Drain

Drain means: finish in-flight steps, take nothing new. Three scopes, all reversible with the matching resume verb:

Scope	Verb	Mechanism
One runner	opschain admin runner drain|resume (POST /api/runners/{id}/drain|resume)	Durable flag on the registration.
A group	opschain admin runner-group drain|resume (POST /api/runner_groups/{name}/drain|resume)	Durable flag on the group row; every member reads as draining.
The fleet, from one replica	POST /api/fleet/drain / POST /api/fleet/resume	In-memory on that replica: stops its offers and directs in-flight runners to finish and stop acquiring. A restart clears it.

Runner and group drains are enforced at the offer boundary on every replica — including a drain requested while the runner sat parked in its poll window, which is re-checked after the poll unparks, so it takes effect at the next offer rather than one poll later. Queued entries whose every eligible runner is draining read every eligible runner is draining on the fleet queue.

Runner groups

A runner group is the durable row registration tokens reference and enrolment inherits, carrying the placement contract (PUT /api/runner_groups/{name}, opschain admin runner-group set):

Attribute	Meaning
name	Letters, digits, ., _, - only (the registered-name charset).
description	Free text.
attributes	Registered key=value placement attributes (e.g. location=dc1) that selector demands match against. Keys must exist in the attribute registry; a key with enumerated values only accepts one of them.
bindings	Absolute node-path prefixes: work targeting a bound subtree is offered only to bound groups' runners (and a bound path constrains the group set at enqueue, so bound work never escapes its zone).
exclusive	The group's runners accept only work targeted at the group — naming it, bound to it, or carrying selectors its attributes satisfy — never unconstrained overflow.
drain_requested_at	The group drain flag.

GET /api/runner_groups/{name}/demand is the autoscaler signal: queued work the group's placement posture could take, plus work its runners currently hold ({"group": …, "queued": N, "running": N}).

DELETE /api/runner_groups/{name} is cleanup for groups nothing can enrol into any more: it is refused while any registered runner or redeemable registration token still references the group, naming each blocker — runner group "x" is still referenced by N registered runner(s)/agent(s) — drain and stop them, then 'opschain admin runner forget', and/or N still-redeemable registration token(s) — let them expire or exhaust before deleting the group.

The attribute registry (GET/PUT/DELETE /api/runner_attributes/{key}, opschain admin runner-attributes) is the admin-managed dictionary group attributes and placement selectors are validated against: unregistered keys are rejected at write time, and a key with an enumerated value set only accepts those values. It is seeded with the reserved keys location and network, which cannot be deleted (attribute key is reserved); their values are unrestricted until an operator enumerates them.

How placement consumes the registry

Every acquire poll presents the runner's posture to the dispatch plane as an acquirer: its id, group, labels, advertised executors, its group's attributes and the group's exclusive flag. A queued step's requirement is matched against that posture — and the fleet queue's verdict ladder uses the same registry facts (registration, liveness, drain) to explain every entry nothing can take. The requirement itself is resolved at enqueue from the request, node settings and the groups' bindings — the full ladder is on opschain docs fleet-queue; the matching mechanics are on opschain docs dispatch.

Events vocabulary

Fleet lifecycle events, recorded on /platform and visible in opschain events and the Activity feed:

• info:runner:registered — enrolment (runner id, name, group, ephemeral).
• info:runner:goodbye — the graceful exit.
• info:runner:forgotten — an admin forget, with the acting principal.
• info:runner:drain_changed, info:runner:group_drain_changed — drain requested or lifted, with the actor.
• info:runner:group_deleted
• info:runner:fleet_quiesced, info:runner:fleet_resumed, warn:runner:fleet_drained — the per-replica fleet verbs (opschain docs platform-halt).
• warn:runner:aged_out — a registered runner went silent past the liveness window without a goodbye; once per offline episode.

Failure modes

• A runner that vanishes silently reads offline within 3 minutes, its queued work explains itself on the fleet queue, its in-flight lease expires into runner_lost, and warn:runner:aged_out announces the episode once.
• A superseded registration (same keypair re-enrolling) is not an error: the old row and its job bindings are replaced — names never collide because the key, not the name, is the identity.
• A revoked runner cannot sneak back: the key's tombstone is durable, and re-registration with it is refused.
• Forget is refused, never forced, while the runner is online or holds a live job binding — drain it and let the work finish first.
• An exclusive group starves unconstrained work by design: an all-zero requirement reads unsatisfiable when the only online runners sit in exclusive groups.

Related pages

• opschain docs fleet-queue — the queued-with-reason surface over these facts.
• opschain docs dispatch — the plane that matches and leases.
• opschain docs platform-halt — the fleet-wide and estate-wide stop verbs.
• opschain docs cli/admin-runners-list, opschain docs cli/admin-runner-show, opschain docs cli/admin-runner-drain, opschain docs cli/admin-runner-forget, opschain docs cli/admin-runner-group-set, opschain docs cli/admin-runner-group-demand, opschain docs cli/admin-runner-attributes-set, opschain docs cli/admin-runner-token-create — the admin CLI surfaces.