The agent plane
packs:agents
The agent plane runs AI (and other automation) agents against OpsChain under explicit, auditable governance. An agent is outbound-only: it registers once, holds an agent-initiated channel open to the server, and serves its capabilities through that tunnel — it binds no listener and needs exactly one egress and one credential. Everything the agent is allowed to do is decided server-side from trust grants; the agent's payload can never name a tier, waive a gate or supply a self-computed identity.
Two separable control surfaces: opschain admin agent manages whether an
agent runs (drain/halt/revoke/forget); opschain admin trust manages what
it may do (grants, assignments, approvals). See
opschain docs cli/admin-agents, cli/admin-agent and
cli/admin-trust.
Naming note:
GET /api/trust_storeis the TLS certificate trust store (public CA material, thecertificatestable) — a different subsystem that merely shares the word "trust".
Registration and identity
An agent enrols by redeeming an agent registration token — minted by
opschain admin agent-token create (cli/admin-agent-token-create),
scoped at mint time to kind agent, a project, a fleet group, optionally
a fixed agent code, with a TTL and max-uses. Possession of the token
cannot enrol outside its scope. admin agent-token list lists live
tokens.
Enrolment registers the agent's Ed25519 keypair (the runner
credential ladder — no long-lived bearer tokens): the identity persists in
the agent's state dir as agent.json plus the private key in its own
0600 agent.key. Sessions are short-lived tokens obtained by signing a
server challenge with that key, renewed automatically before expiry. The
server signals an unrecognised agent with a re-register code, and the
agent re-registers with the same durable keypair when it still holds a
registration token. An identity registered with one server refuses to be
presented to another (a named error pointing at --dir).
Agents are rows of the unified fleet registry (kind agent,
alongside runners), identified by member id, with a per-project-unique
code and a fleet group. The code names the agent — the logical
identity; each enrolled process is one of its instances (a fleet
member with its own keypair, bound to the release it runs, labelled with
its lane), and a second state directory with the same code enrols a
second instance — see opschain docs agent-releases for instances,
lanes and the release rollover. Registration and session establishment are
gated on the packs:agents entitlement (or the bare agents feature
it supersedes; see opschain docs entitlements), with the
licence's agents limit bounding concurrently connected instances
(existing connections stay live on lapse or a shrunk
limit; a new registration or reconnect refuses loudly with a named error
plus a warn:licence:feature_refused event). The manifest may declare a
type (typevocabulary: service or ai): an
ai-type enrolment additionally hard-depends on packs:agent-governance
packs:ai-agents, refused naming every missing pack (opschain agent dev --type aiand the SDKs' type options declare it). Registration is also refused while the agent plane is halted or the system is quiesced.
The service account
Every registration mints (find-or-create) a durable service-account
principal: a users row with the distinct auth_provider = 'agent'
identity kind, the deterministic username
svc-agent-<project>-<code>, no password and no interactive login — its
only credential is the channel's keypair/session handshake, and it can
never start a session or mint an API key. It survives re-registration and
key rotation, and is addressable by policy assignments like any
principal. On the identity surfaces the row badges agent beside the
human kinds, carries the fleet cross-link (agent: {project, code, member_id} on the user listings, with service_account_id/_username
on the fleet detail pointing back), rides the open-set provider=agent
filter, and is dormant once its agent is revoked or forgotten — out
of pickers by default, visible on request, exactly like a dead-provider
shadow. It is never managed through the human user lifecycle, nor
through the service-account verbs that manage 's human-minted
automation principals (auth_provider = 'service_account',
opschain docs cli/admin-service-accounts) — fleet verbs are its
lifecycle.
The owning human or team
Every agent names its owner as first-class data: owned_by
on the manifest — declared at registration (opschain agent dev --owned-by, Go SDK OwnedBy, Python owned_by=) and validated against
the identity catalog as a known user or group; an unknown name refuses
registration. Admins override it with opschain admin agent owner <agent-id> --owned-by <name> (audited as info:agent:owner_changed); a
manifest that declares no owner preserves the stored one, so the override
survives re-registration. The owner shows on the fleet views and the trust
review, and rides the agent's warn events (warn:agent:halted,
warn:agent:request_refused) so escalations route to the owning team.
Ownership is accountability only — it never derives a privilege ceiling or
floor, and offboarding a user with opschain admin users reassign-automations moves the agents they own alongside their triggers
and schedules.
Trust presumes an owner: a trust grant (or raise — every raise is a grant
write) naming supervised or autonomous for an unowned agent refuses
by name at grant time, with the one-verb fix in the refusal body —
assign the owner, then retry the grant. advise carries no such
requirement. Existing grants are untouched: an agent that lost or never
had an owner keeps working and is instead flagged as unowned on the fleet
list, the agent sheet and the project agents view, each of which also
filters to unowned agents. A composition without an identity backend
cannot hold a validated owner, so the requirement does not apply there.
The GUI assigns and clears owners in place (the agent ops sheet and the
project agents view, under the same role that may drain or halt), riding
the owner override — on v2, AgentService/SetAgentOwner, the
POST /api/agents/{id}/owner twin.
Signed releases — enrolling from an AgentRelease
An agent ships as a signed AgentRelease: an
immutable <name>@<version> row holding the frozen manifest document, the
content-addressed artifact identity (git sha or image digest) and an
Ed25519 signature over both, verified against the registered signing-key
registry (opschain docs cli/admin-agent-release). Releases promote one
rung at a time up the estate's stage ladder (default dev → test → prod); the signature re-verifies at every promotion, and on a
fabric-carrying estate a promotion into a governed rung is held in the
same approval funnel governed change enters. The stage then travels with
the instance as a principal attribute: an environment's release-stage
floor and a grant's release_stage condition decide where that build may
act (opschain docs agent-releases, "Where a stage may act").
A manifest that names a release (release_name/release_version; Go SDK
ReleaseName/ReleaseVersion, Python release="name@version") is
verified at registration and refused by name when the release is unknown,
unsigned, tampered (the stored content no longer matches its
signature), of another declared type (a signed release
cannot be misdeclared), or below the estate's agents.release_stage
floor. A verified enrolment binds the release to the instance: the fleet
detail and the trust review answer "which release is this instance
running, at which stage", and two instances of one agent may run two
releases — the shadow lane beside stable, the canaries of a
rollover. An agent that names no release enrols exactly as before; a
retired or revoked release refuses enrolment by name. Changing the
release an instance runs is opschain admin agent rollover
(opschain docs agent-releases).
Manifest disclosures and the type attestation
The manifest discloses what the agent is and does. Every type may declare a purpose — the "what is this
for" label. The ai type must additionally disclose, at registration:
- model identification —
model_provider,model_name(and an optionalmodel_version); - adapter/SDK identification —
adapter(optionaladapter_version), e.g.opschain-claude-adapter; - tool surface —
tool_surface, the estate tools the agent wields (e.g.request_lane,docs,asker_reads), stored sorted and deduplicated; - custody mode —
custody_mode, one of the two modes:agent-hostorvault-to-host. Any other value refuses at the manifest edge; the credential itself never appears in the manifest.
Declare them with opschain agent dev --purpose/--model/--adapter/ --tool-surface/--custody, the Go SDK Disclosures option, or the Python
disclosures= dict; the Claude adapter attaches its own disclosures
automatically when the agent declares none and ships without a release.
Disclosures render on the fleet detail (disclosures, with
disclosures_from_release when frozen — see below), the trust review and
opschain admin agent show (PURPOSE/MODEL/ADAPTER/TOOL SURFACE/CUSTODY/
DISCLOSED BY).
The type attestation binds the declared type to the enrolment: an ai-type manifest missing any required AI
disclosure refuses by name, and a manifest of any other type declaring
them refuses as a type contradiction — both land the audited
warn:agent:type_attestation_refused event. The same contradiction is
detected at the adapter seam: adapter-transit transcripts arriving from
an agent not declared ai refuse and land
warn:agent:adapter_transit_refused. Where the agent enrols from a
signed release whose frozen manifest document carries a disclosures
object, the frozen set is authoritative and immutable: the live
views re-read it from the release row, and a registration declaring a
differing set refuses by name (change what an agent disclosed by
shipping a new release). What stays honestly out of technical reach: the platform cannot classify arbitrary customer code
that never transits its seams — a self-authored LLM agent misdeclared as
service that avoids the platform adapter is an audit and licence
compliance matter (posture), not a technical enforcement.
The manifest and capabilities
At registration the agent declares its manifest: spec version
(current 1.0; the server bounds accepted versions between a floor and a
target), agent name/version, and its capabilities as name → JSON config.
The runtime supplies the mandatory five built-ins itself — healthz,
readyz, info, metrics (Prometheus), logs (a bounded 500-line
ring) — so a customer agent only ever writes its real endpoints. Optional
capabilities (chat, config, x-* namespaces) are served by an
in-process handler or reverse-proxied to a supervised local process
(--target, loopback only; opschain agent can also supervise and
restart the child itself, handing it its port via
OPSCHAIN_AGENT_PORT/PORT). chat defaults to
{"streaming":true,"timeout_seconds":120}.
The channel and the tunnel
The channel is agent-initiated: a Connect bidi stream (h2c against a
plain-HTTP server), falling back automatically to long-poll
(--transport auto|stream|poll) — the long-poll carriage is fully
supported, not degraded. Reconnects use exponential backoff with full
jitter (1 s → 60 s); the heartbeat (the stream's 60 s keepalive, or each
long-poll) refreshes the agent's durable channel-ownership record, so
online status is estate-wide: every server replica answers
GET /api/agents, the detail view, the project-plane rows and the
governance dashboard's fleet counters from the same record — a
connected agent reads online (with its carriage) whichever replica
serves the read, a record not refreshed within its two-minute freshness
bound (twice the slowest heartbeat cadence) reads offline honestly, and
a clean shutdown's goodbye marks the agent stopped everywhere at once.
A silent agent ages out.
Inbound, the server materialises every registered agent's declared
surface under /api/projects/{project}/agents/{selector}/{capability}
(GET and POST) — the ordinary API, ordinary caller authorisation: the
decision walks the agent's governance path /{project}/agents/{code},
with read sufficing for the five minimum surfaces and execute required
for every interactive capability (chat, config, x-*). Deny renders 404,
so an unauthorised caller cannot distinguish a denied agent from an
absent one, and an agent cannot be probed for undeclared surfaces.
Selector resolution
order: exact member id (one instance, addressed directly), then
per-project code — the agent, routed to ONE of its instances: the
lane the request names in X-Opschain-Agent-Lane (default stable),
then the least-loaded connected, readyz-green instance of that lane whose
cached release stage passes admission; a shadow-lane instance beside
it receives the same dispatch with its answer discarded — then group
name (routed to one healthy member — channel-connected, readyz-green,
least-in-flight). The answering member is named in the
X-Opschain-Agent-Id response header, the exchange in
X-Opschain-Exchange-Id. Proxied exchanges are bounded: a 30 s default
per-exchange deadline (a capability config may raise it, never past the
server's 10-minute hard ceiling), at most 16 concurrent exchanges per
agent, 8 MB request bodies (413 above), and a chunk-credit window that
caps per-exchange server memory. An edge refusal is audited as
warn:agent:tunnel_denied. Chat exchanges are captured as transcripts
(log segments keyed to the agent, one sealed segment per exchange) and
read back at …/chat_transcripts/{exchange}. GET …/chat_transcripts
(no exchange id) lists the caller's own sealed exchanges with the
agent, newest first — exchange id, started/sealed timestamps and a
redacted first-line preview per row (?limit= default 50, capped at
200; ?cursor= keyset paging) — filtered server-side by the asker
principal recorded on the exchange, never anyone else's; authorisation
is the same read decision as the per-exchange transcript read, and an
aged-out exchange leaves the list because the evidence itself is gone.
AI-type agents additionally
ship adapter-transit transcripts — every model interaction crossing
the Claude adapter — over AgentTranscriptService/AppendAdapterTranscript;
the server redacts each line (secret-shaped values, credentials, email
addresses) before persistence, seals the exchange on the final batch
(audited info:agent:adapter_transcript), and serves it back at
…/adapter_transcripts/{exchange} (see opschain docs ai-agents).
The request lane: agents calling the platform API
Call on the agent RPC is the agent's one way to call the platform API:
GET and POST only, /api/... paths only, bodies bounded. The server
dispatches the call in-process through the composed HTTP surface under
the agent's service-account principal — so authorisation, request logging
and tier enforcement apply exactly as for any caller — and writes the R16
audit event itself (info:agent:request, or
warn:agent:request_refused), never the agent. The audited payload
carries ids and codes only, never usernames or emails.
Run-as (acting on behalf of a user)
A call may name a run-as mode — never an identity the agent computed.
The server resolves it against an explicit act_as-class trust grant
covering the call's governance target ("owner" resolves the target
asset's designated owner from the properties key named by the
agents.owner_property_key config, default metadata.asset_owner; a
username grant resolves statically). The acted-as principal's own
authority is the ANDed ceiling, resolution failures are named refusals
(never a silent fallback), and the acted-as identity plus how it was
resolved land in the audit event and the run's metadata. The governance
target is derived server-side from the request itself (POST /api/changes, POST /api/workflow_runs and their node-path variants).
A third mode serves chat answers: asker:<exchange-id> resolves to the authenticated user whose live chat exchange the agent
is currently serving — the server stamps the exchange id onto the proxied
chat request (X-Opschain-Exchange-Id, any caller-supplied copy
stripped) and keeps the exchange → caller record only while the exchange
is open. On a multi-replica estate the record lives with the exchange:
a delegated read served by another replica resolves the asker through
the exchange-holding replica, so the mode needs no sticky topology. The mode is read-only (GET), requires an explicit
act_as:asker trust grant covering the read path (--act-as asker), and
runs the read under the asking user's own principal, so the answer can
never widen the asker's view; the audit event records the acted-as
principal and the chat-exchange asker via grant … @ exchange … source.
Governance: tiers, grants, assignments
The governance fabric sits behind the packs:agent-governance
entitlement — engine in base, pack gates creation and
surfaces. Gated (each a named 403 refusal): trust-grant create, approve
and reject, the grants listing, the recommendations lifecycle, and the
per-agent activity and trust-review reads (REST and the v2 twins alike).
Base and never gated: tier enforcement on the request lane (existing
grants keep governing on lapse), assignments — the ADVISE floor's carrier
— grant revoke (shedding authority is never licence-gated), the halt
scalpels and the audit trail.
The funnel's reads are not admin-only: a path-RBAC rule granting
read on /governance/agents admits a governance role to the agent
trust surface read-only — the fleet and agent detail (disclosures and
owner included), the release catalog, the grants listing with approval
state, assignments, activity and the trust review — without admin
status, pack gates unchanged. The GUI renders this as the read-only
Governance → Agents view. The admin verbs (drain, halt, forget, revoke,
owner, the trust decisions) stay on the /admin updatable
determination.
A trust grant is deliberately the authz-rule shape — (subject, node-path scope, action patterns) — with the decision widened from allow/deny to a tier:
| Tier | Meaning |
|---|---|
advise | May only produce recommendations. |
supervised | Executes behind a server-injected approval gate. |
autonomous | Executes ungated. |
No matching grant means the request is refused — ADVISE is the lowest
grant, not the default. Subjects: one agent's service-account principal
(service_account), every agent in a named group (agent_group), or
every agent whose group carries a registered attribute value
(attribute — trust policy that survives fleet churn). Scopes are
node-path globs covering their whole subtree, so an asset created
tomorrow is covered by today's grant. Grants may carry their own
approver list, an act_as class, an expiry, and a lifecycle status
(active, pending, revoked). Grants compile into a version-keyed
in-memory snapshot (tier resolution is a trie walk, no database on the
decide path), invalidated wholesale on every write.
Resolution for one (path, action): grants whose scope covers the path and
whose action pattern matches, ranked most-specific-scope first,
least-permissive tier among ties. The grant outcome then composes with
the agent type's tier ceiling: a manifest
type with a ceiling row never acts above it, whatever the grants say —
effective = min(grant tier, type tier_ceiling). The ai type
ships ceilinged at supervised; opschain admin trust ceilings lists the rows and opschain admin trust ceiling <type> --ceiling <tier> writes one (step-up floored, pack-gated, audited as
info:trust:ceiling_set).
Node-classification ceilings (over metadata)
compose the same way from the other axis: a rule keys on metadata
predicates (e.g. core.criticality=critical), and at every target
node whose effective classification matches, the agent's tier is
additionally capped — effective = min(grant tier, type ceiling, every matching node ceiling), multiple matching rules resolving min-wins.
Matching evaluates the compiled effective-metadata snapshot at the frozen
target on each request, so a reclassification moves the clamp on the next
request — "autonomous in Dev, supervised wherever critical" is
classification policy, not per-subtree grant bookkeeping. opschain admin trust node-ceilings lists the rules and opschain admin trust node-ceiling set|remove writes them (the same step-up floor
trust.ceiling_write, pack gate and audit discipline, as
info:trust:node_ceiling_set/info:trust:node_ceiling_removed with the
rule before and after).
A clamped decision stamps grant_tier, tier_ceiling (the type
conjunct) and node_ceiling/node_ceiling_ids (the matching rules)
beside applied_tier in the run metadata and the request event, so the
audit explanation is always spelled out.
Grants also carry the cap fields — rate (governed
work requests per minute), budget (invocations per UTC day) and
tokens (model tokens per UTC day spent through the model gateway), 0
meaning capless — rate and budget enforced as O(1) counter reads at the
same chokepoint as tier resolution, the token budget enforced pre-flight
at the model gateway from usage figures read off the provider's own
responses. An over-cap request refuses loudly naming the grant,
the cap and the window reset, and lands the audited
warn:trust:cap_refused event.
On a mutating agent call that creates work, the resolved tier is enforced server-side:
- autonomous — passes clean.
- supervised — the un-bypassable root approval gate is injected.
Approvers resolve in order: the winning grant's approvers → the
target's designated owner (the owner-metadata convention) → the
agents.approversconfig fallback. With none of the three:Agent-created changes hold for approval, but no approver is configured: set an approver on the trust grant, an owner in the asset's owner metadata, or agents.approvers in the server configuration. - advise — the mutating call is refused:
Trust tier ADVISE on <path>: this agent may only recommend here — file a recommendation via POST /api<project>/recommendations instead. An instance serving theshadowlane is forced to this outcome whatever its grants say (the refusal names the lane): its recommendations carrylane: shadowand link to the stable lane's proposal for the same incident — a second proposal for an open (agent, subject node path, correlation root) key lands as aduplicateof the open card, which shows both provenances; a duplicate is never accepted on its own.opschain recommendations list|show <project>renders the cards with their lane, correlation id and linked duplicates (opschain docs cli/recommendations). - no grant —
No trust grant covers <path> for action "<action>": this agent is not trusted to request work here (grant trust via the trust-grant admin API).
The agent attribution (agent_id — the fleet member id, agent_instance_id
— the acting instance's id, agent_code, agent_project,
requested_by_service_account, applied_tier, trust_grant_ids, and on
delegation on_behalf_of/on_behalf_of_id/resolved_via) is stamped
into the run's stored metadata — durable data, not log archaeology. The
run's own initiator (initiated_by_type: agent, initiated_by_id) names
the instance that raised it: the instance id is the stable identity
across re-enrolment and release restarts, where the member id is the
keypair in hand.
Assets under management
A trust assignment records an agent's inventory — the scope it is the
SME for — and is the natural scope to hang higher-tier grants on. With no
matching grant, an assignment confers the configured assignment
floor: advise (the default) or deny
(agents.assignment_floor in config.yaml — see opschain docs config).
Raising trust is an approval-fabric event
Managing grants is itself a governed action: an agent proposing a grant
change stages it as pending, and only active grants compile. A proposed
revoke leaves the grant active — and compiled — until a human applies or
rejects it. The admin verbs: opschain admin trust grant|list|revoke|approve|reject|assign|assignments|unassign. The human
half — list, approve, reject and the assignment verbs — rides the
API v2 TrustService; grant and revoke stay on the REST routes
(POST /api/trust/grants, POST /api/trust/grants/{id}/revoke) that the
agent-credential lane shares, since the v2 plane authenticates users,
never agent keypairs.
On a composition carrying the approval fabric (the adopted PostgreSQL
schema), every grant write naming supervised or autonomous is a
trust raise and enters the same pending_approval funnel human-created
governed change uses — never a direct edit. The
write lands the grant pending and opens a tier-sourced strict
checklist (quorum 1, self-approval off, the grant id as the subject):
approvers resolve from the grant's own approvers, else the
agents.approvers config — with neither the write refuses by name, since
a governed raise is never approver-less. The raise appears in the one
approvals inbox (opschain approvals list); opschain approvals approve <grant-id> activates the grant, approvals reject revokes it, and an
unanswered quorum TTL lapses it to revoked — an unapproved raise never
takes effect. While the ceremony is open the direct
admin trust approve|reject verbs refuse by name — the typed
failed_precondition pointing at the approvals inbox.
Events: info:trust:raise_held at creation, info:trust:grant_approved
or info:trust:grant_rejected at the decision. Without the fabric
composed, grant writes keep the base-posture behaviour above.
The trust review
GET /api/agents/{id}/trust-review (opschain admin agent trust-review)
is the decision-support view before promoting or demoting an agent:
requests by applied tier, requests refused, recommendation outcomes by
status, the active grants applying to its governance subject, its
assignments, the server's assignment floor, and — when set — the agent's
owned_by and its type's tier_ceiling. GET /api/agents/{id}/activity is the raw audit timeline, GET /api/agents/{id}/assignments its inventory.
Lifecycle control
Admin-gated fleet verbs (GET /api/agents, GET /api/agents/{id}, and
the actions below; the detail view fingerprints the public key, never
renders it raw):
| Verb | Effect |
|---|---|
| drain / resume (`POST /api/agents/{id}/drain | resume`) |
halt / resume (POST /api/agents/{id}/halt, …/halt/resume) | The per-agent emergency scalpel: hard stop — the durable halt flag flips, channel and sessions are severed immediately, every credential refused until an explicit resume. Reversible by design, no tombstone. A reason is required: a reason is required: the halt verb is break-glass audited. Group form: POST /api/agent_groups/{name}/halt (runner members of the group are unaffected). cli/admin-agent-halt, cli/admin-agent-halt-resume. |
revoke (POST /api/agents/{id}/revoke) | Tombstones the agent's key and severs its channel and sessions; the identity stays refused until re-enrolled with a fresh keypair and token. A revoked id reads as 404 — the tombstone stays un-exposed. cli/admin-agent-revoke. |
forget (POST /api/agents/{id}/forget) | Removes a stale registration so the identity may re-register later (not revocation). An online agent is refused: agent is online: stop or drain it before forgetting it (409). cli/admin-agent-forget. |
rollover (v2 AgentService.RolloverAgent / AdvanceAgentRollover) | Moves one lane of the agent to another release: drain each instance, instruct its restart over its channel, resume on re-enrolment; replace or canary=<n> with continue / abort / rollback / finish. blue-green restarts nothing: a green lane comes up on the release beside the serving lane (licence headroom for the overlap checked first), continue flips dispatch to it in one relabel, rollback flips back, finish drains, halts and retires the old lane. cli/admin-agent-rollover, cli/admin-agent-rollover-show, opschain docs agent-releases. |
Plane-wide, opschain admin platform halt --plane agents severs every
channel and suspends every service-account credential (the direct-HTTPS
gate answers agent plane halted (posture <id>): <reason>) until a
platform resume — see opschain docs platform-halt.
What the agent sees — platform-driven state changes
Every state the platform imposes on an agent is something the agent
itself observes, logs and reports — an operator reading the agent's own
log file, its logs surface (GET …/agents/{code}/logs, the GUI's
capability panel) or its readyz/info surfaces sees what happened to
it and why, without consulting the server.
Typed reasons on the wire. Every refusal the agent plane answers for
a platform-driven state carries a machine-readable reason beside its
human text: an ErrorInfo detail (domain opschain.agent.v1, with
metadata) on the agent-protocol error, mirrored in the
Opschain-Agent-Reason response header; the capability proxy's 503
body is JSON carrying the same reason, message and metadata
(Content-Type: application/json, the reason also in the header); and
the model gateway's refusal JSON carries error.reason. Additive —
clients that ignore them see the same Connect codes and texts as before.
| Reason | When | Metadata | The agent… |
|---|---|---|---|
AGENT_PLANE_HALTED | the agent plane's emergency posture is active | posture_id, reason | waits |
AGENT_HALTED | this agent, or its group, is halted by the scalpel | reason, scope (agent / group) | waits |
PACK_ENTITLEMENT_REQUIRED | the licence no longer grants a pack the agent needs (packs:agents; for the ai type packs:agent-governance + packs:ai-agents) | feature, tier | waits |
LICENCE_LIMIT_EXCEEDED | the connected-agents limit is full | limit, count, tier | waits |
SYSTEM_QUIESCED | the maintenance quiesce barrier is closed | reason | waits |
AGENT_UNAUTHENTICATED | the presented session is not live (expired, severed) | — | re-handshakes silently |
AGENT_REREGISTER_REQUIRED | the server no longer knows the registration | — | re-registers with its durable key |
AGENT_REVOKED | the identity is tombstoned | — | exits, naming the reason |
AGENT_RELEASE_REVOKED | the release this instance runs was revoked; the instance is halted through the per-agent halt | release, reason | waits, until rolled to a current release and resumed |
AGENT_INSTANCE_LIVE | a fresh identity key presented the id of an instance that is still live under another key (the one already_exists left) | instance_id | exits — enrol without the instance id to start a second instance |
AGENT_UNREACHABLE / AGENT_BUSY / AGENT_DRAINING | the proxy's own 503 classes (no live channel; concurrent-exchange bound; only the minimum spec while draining) | last_seen_at for unreachable | — (caller-side) |
MODEL_CALL_REFUSED | the model gateway's own refusal (unknown, disabled or unreferenced provider, missing credential, token cap) | — | — (the model SDK surfaces it) |
A platform instruction reaches the instance at once on the replica that
holds its channel. The instruction (a rollover's restart_from_release)
travels down the channel as an exchange the moment it is issued: it is
pushed onto an open stream, and a parked long-poll is completed with it
rather than waiting out its window. The runtime acknowledges it and arms
its drain, then acts on the drain at its next carriage turn — and the
server supplies that turn the moment it has the acknowledgement: the
long-poll the runtime re-parked in the meantime is completed (empty, "poll
again"), an open stream is ended cleanly, so a runtime that reads its
armed state only between polls — every long-poll runtime, the Python SDK
included — restarts within milliseconds rather than up to one poll window
(50 s by default) later. On an estate of several replicas this is
replica-local by design: the replica holding the channel issues the
instruction and gives the turn; a replica that does not hold the channel
issues nothing and signals nothing across replicas, so a rollover started
on another replica waits for the holding replica's next sweep (seconds)
and then proceeds at the same pace. Halts, revocations and the plane halt
reach a connected agent the same way, below.
The channel closes with the reason. When an agent is halted, its
group or the plane is halted, its identity is revoked, or a licence
change withdraws a pack it needs, the server does not just drop the
connection: the bidi stream ends with the typed refusal and a parked
long-poll answers it at once, so a connected agent learns the state the
moment it is imposed, not at its next handshake. A licence change is
reviewed on every server replica (packs:agents withdrawn → every
connected agent; the ai dependency withdrawn → every connected
ai-type agent); a lowered connected-agents limit is not swept —
existing connections stay live and the limit bites at the next
handshake.
Wait, never die. Every platform-driven state above except revocation
is a wait: the agent keeps its identity, logs the transition once, and
retries the channel under its bounded backoff (1 s → 60 s, full jitter)
until the platform resumes or re-entitles — then logs the restoration.
This includes a re-registration refused for one of these reasons: the
identity directory is kept and the registration retried. Only
AGENT_REVOKED (and an unrecoverable registration refusal, such as a
bad token) ends the process, and the last log line names why. A fresh
identity registering an already-registered code is simply a second
instance of the agent; already_exists survives only for a fresh
keypair presenting the id of an instance that is still live — stop
that process, or enrol without the instance id. A platform-instructed
restart from a release (restart_from_release, the rollover verb) is
not a refusal: the runtime acknowledges it, drains, says goodbye and
restarts itself (exec mode) or exits with code 75 for its process
manager (exit mode), then re-enrols the same instance from the new
release.
One line per transition. The Go runtime (opschain agent) and the
Python SDK log the same lines, exactly once on entering a state — never
once per backoff tick — to the process log and the logs surface:
channel open (stream) ← first connection (or "long-poll")
platform: agent plane halted — <reason> (posture <id>); channel closed, retrying every ≤60 s until resumed
platform: agent halted — <reason> (operator); channel closed, retrying every ≤60 s until resumed
platform: agent group halted — <reason> (operator); channel closed, retrying every ≤60 s until resumed
platform: entitlement <pack> withdrawn — <text>; channel closed, retrying every ≤60 s until re-entitled
platform: licence limit reached — <text>; retrying every ≤60 s
platform: system quiesced for maintenance — <reason>; retrying every ≤60 s until resumed
platform: server unreachable — <error>; retrying every ≤60 s
platform: channel lost — <error>; retrying every ≤60 s
platform: session refused repeatedly — <text>; re-handshaking retrying every ≤60 s
platform: channel refused — <text>; retrying every ≤60 s ← an unclassified refusal
platform: channel restored after <duration> (stream|long-poll) ← recovery from any of the above
platform: agent release revoked — <text>; channel closed, halted until an operator rolls this instance to a current release and resumes it, retrying every ≤60 s
platform: restart from release <name@version> instructed (<mode> mode); draining, then restarting ← a rollover
platform: agent revoked — <text>; exiting ← the only fatal line
platform: re-registration refused — <text>; exiting ← an unrecoverable registration refusal
A halt whose reason changes (a second halt with a different reason) is a new transition and logs again; a severed session's single re-handshake is silent (the handshake answers the real reason), and only a repeatedly refused handshake becomes a logged state.
On the surfaces. readyz carries checks.platform (the state name)
and a platform object — state (connected, halted:plane,
halted:agent, entitlement_withdrawn, licence_limit, quiesced,
unauthenticated, unreachable, refused, revoked, or starting),
detail (the posture id, the pack, the halt scope and reason), since
and, when connected, carriage; info carries the same platform
object. The Python SDK exposes it as agent.platform_state().
What changes behaviour, and who owns it
| Lever | Authority | Effect |
|---|---|---|
| Trust grants / assignments | Admin API (opschain admin trust …) | The tier, approvers and act_as class per (subject, scope, action). |
agents.approvers | config.yaml (opschain docs config) | Fallback approvers for SUPERVISED outcomes. |
agents.assignment_floor | config.yaml | advise (default) or deny with no matching grant on assigned scope. |
agents.owner_property_key | config.yaml | The properties key naming an asset's designated owner (default metadata.asset_owner). |
| Registration tokens | opschain admin agent-token create | Who may enrol, into which project/group/code, for how long. |
| Drain / halt / revoke / forget | Admin API (opschain admin agent …) | Whether the agent runs at all. |
The packs:agents entitlement + agents limit | The installed licence (opschain docs entitlements) | Whether agents can register and how many may be connected at once; packs:agent-governance opens the grant/recommendation/review surfaces, and with packs:ai-agents admits AI-type enrolments. |
Events vocabulary
All searchable via /api/events, carrying the agent's governance path
/{project}/agents/{code}, emitted server-side at the enforcement seam —
no agent code path can omit its own trail:
- Lifecycle:
info:agent:registered(with theinstance_id,laneand release stage),info:agent:goodbye,info:agent:forgotten,info:agent:revoked,info:agent:drain_changed,warn:agent:halted,info:agent:halt_resumed,warn:agent:group_halted,info:agent:group_halt_resumed - Rollover:
info:agent:rollover_started,info:agent:instance_drained,info:agent:green_ready,info:agent:rollover_flipped,info:agent:instance_restarted,info:agent:canary_held,info:agent:rollover_continued/rollover_aborted/rollover_rolled_back/rollover_finished,warn:agent:rollover_refused— one correlation chain per rollover - Channel/tunnel:
info:agent:channel_opened,info:agent:channel_closed,warn:agent:tunnel_denied,info:agent:chat - Request lane:
info:agent:request(service account, method, path, status, applied tier, acted-as principal),warn:agent:request_refused - Governance:
info:trust:granted,info:trust:grant_proposed,info:trust:raise_held,info:trust:grant_approved,info:trust:grant_rejected,info:trust:revoked,info:trust:revoke_proposed,info:trust:revoke_rejected,info:trust:assigned,info:trust:unassigned,info:trust:ceiling_set,info:agent:owner_changed,warn:trust:cap_refused - Transcripts:
info:agent:chat,info:agent:adapter_transcript - Recommendations (the ADVISE output):
info:recommendation:filed,info:recommendation:accepted,info:recommendation:dismissed,info:recommendation:expired,info:recommendation:superseded,info:recommendation:duplicate - Licence:
warn:licence:feature_refused(a registration or reconnect refused over thepacks:agentsfeature/limit, or an AI-type enrolment missing its pack dependencies — throttled per subject) - Releases (platform path — releases are estate-global):
info:agent_release:key_registered,info:agent_release:key_status_changed,info:agent_release:created,info:agent_release:promoted,info:agent_release:promotion_held,info:agent_release:promotion_rejected,info:agent_release:retired,info:agent_release:revoked,warn:agent_release:refused; on the agent path:warn:agent_release:enrolment_refused
Failure modes
- An unreachable agent is routed around under group addressing; with members but none healthy the caller sees 503 "unreachable", not "absent". A silent agent ages out of the online view; its identity and grants persist.
- A multi-replica server estate needs no agent stickiness: an agent's
channel terminates on one replica, its owner is recorded durably, and a
capability request (or a long-poll response) landing on another replica
is forwarded to the owner over the replica-to-replica lane — the caller
sees the same answer whichever replica the load balancer picked. An
act_as: askerdelegated read dispatched through another replica resolves the asking user the same way — the exchange-holding replica answers the lookup, so the delegation works without sticky topology and still dies with the exchange. Fleet accountability needs no stickiness either: online status reads the shared ownership record, so every replica's list, detail and dashboard counters agree. Prerequisites: each replica'speer_urlset and the estate-shared token secret (seeopschain docs config); withoutpeer_urlthe non-owning replica answers the honest 503 with the agent's last-seen time. - A misbehaving agent has a graduated response: drain (graceful) → halt (immediate, reversible, audited with a reason) → revoke (credential tombstone). Grants can be revoked independently, effective immediately (snapshot invalidation on every write). The agent itself logs each of these as it lands (see "What the agent sees"): a halt or a withdrawn entitlement is a wait it rides out with its identity kept, a revocation is the one state that ends the process.
- A dead agent cannot pin the server: exchange deadlines, concurrency caps and chunk credits bound every proxied call.
- Governance cannot be bypassed from the agent side: tiers resolve from the compiled snapshot server-side, the approval gate is injected server-side, run-as identities are resolved server-side, and the audit events are written server-side.
Related pages
opschain docs cli/admin-agents,cli/admin-agent,cli/admin-trust,cli/admin-agent-token-create,cli/agent,cli/agent-dev— the command surface.opschain docs entitlements— theagentsfeature and count limit.opschain docs platform-halt— the plane-wide emergency posture.opschain docs runner-registry— the shared fleet registry and groups.opschain docs events— the audit stream the R16 events land in.