Secrets providers

Included from Basic tiersecrets:multi-vaultpacks:secrets-awspacks:secrets-azure

A secrets provider is a registered vault connection — the backend the server resolves secret-vault:// references through. Secret values never live in properties, action files or run documents; everything carries a reference, and the provider is where the referenced value actually lives. The provider is a first-class connection object with the identity-provider treatment end to end: addressed by an estate code, an encrypted, write-only credential (requests carry it, the store encrypts it at rest, and no response, audit row or error ever echoes it — reads serve credential_set), audited writes, per-connection health and a test verb. The hands-on walkthroughs are opschain docs getting-started/secrets-providers (operating the backends) and opschain learn 3.5 (writing and using secrets); this page is the component reference.

The kind set — open and additive

Kind	Backend	Gated by
openbao	OpenBao, over its HTTP API	base platform
hashicorp-vault	HashiCorp Vault, over the same HTTP API	base platform
aws-secrets-manager	AWS Secrets Manager	packs:secrets-aws
azure-key-vault	Azure Key Vault	packs:secrets-azure

The vault kinds share the OpenBao/Vault wire protocol today but stay distinct kinds, so posture, defaults and future divergence are per-kind — the set is open by construction, and a new backend arrives as a new kind on the same connection object, CRUD verbs, test verb, health recording and audit events. The cloud kinds are pack-gated: creating one, or enabling a disabled one, answers the typed ERROR_REASON_FEATURE_ENTITLEMENT_REQUIRED refusal naming the kind's pack (see entitlements). Their kind-specific configuration — field meanings, secret mapping, IAM/Entra ID requirements — lives on their own pages: AWS Secrets Manager and Azure Key Vault.

The record, attribute by attribute

Attribute	Meaning
code	The unique estate identifier the provider is addressed by (lowercase letters, digits, -, _, starting alphanumeric). Immutable after create.
display_name	The human name the GUI renders.
kind	The backend kind (table above). Immutable after create.
kv_version	The KV secrets-engine version, 1 or 2 (vault kinds).
address	The vault endpoint URL (azure-key-vault: the vault URL, required; aws-secrets-manager: an optional endpoint override — empty derives it from the region).
region / tenant_id	The cloud kinds' location facts — refused on the other kinds (see the pack pages).
auth_method	token | userpass | approle (vault kinds), access-key (AWS), client-secret (Azure). The method's secret half rides the write-only credential seam.
username	The non-secret identity half: the userpass username, the IAM access key id, or the Entra ID application (client) id.
role_id	The AppRole role id (not a secret; the secret id is the credential).
credential_set	Whether a credential is stored; the value itself is write-only.
credential_ref	A provider-qualified secret-vault://<code>@<path>/<key> reference the auth credential lives at — an address in ANOTHER enabled provider's vault, never a value. Empty on a record whose credential is stored inline (encrypted at rest). A vault cannot custody its own key.
mount_path	The KV engine mount path (empty: the engine default); for the cloud kinds, an optional secret-name prefix.
ssl_verify / ssl_ca_cert	The connection's TLS posture: verification on by default, with an explicit opt-out and a trusted CA file path.
cipher_stacking	Whether values written through /api/secrets/store are encrypted again by the server's codec on top of the backend's at-rest encryption (default true).
password_length	Length of server-generated secret values (0: the endpoint default).
enabled	Whether the provider resolves. A disabled provider resolves nothing and says so.
resolution	Where references routed to this provider are read: server (the default — the server resolves them before dispatch, as every provider did before) or runner (the runner and the agent SDK resolve them at Prepare from their own secrets_providers configuration; the server stores no credential for such a provider, never reads it, and refuses its own *_ref fields that route to it). See "Where references resolve" below.
rate_limit_reads_per_second / rate_limit_mints_per_minute	The provider's rate budget: the KV reads per second and the vault token mints per minute the platform meters its own calls to this provider at — on the server for a server provider, on every runner for a runner provider (the budget rides the job spec with the deferred references). Per lane: a positive integer is this provider's own budget; 0 inherits the estate default secrets.default_rate_limit (opschain docs settings-reference), itself unset = unbounded; -1 is unlimited at this provider even when the estate default is bounded. Anything below -1 is refused by name. Steps wait for budget in a bounded queue rather than bursting; see "How steps consume it".
last_test_at / last_test_success / last_test_error	Per-connection health, recorded by the test verb — the failure reason never carries credential material.

Listings order by code ascending — there is deliberately no display order, because providers never render as an end-user picker.

Lifecycle

• Create registers a connection: code and kind immutable, the credential supplied through an environment variable or a no-echo prompt (never argv), encrypted at rest. An approle provider may defer its credential and seeds disabled until one is rotated in.
• Update partially updates — absent fields preserve stored values; a supplied credential rotates the stored one (audited, never echoed), and a supplied credential_ref re-pins the reference and clears any inline credential — custody moves to the named provider's vault.
• Enable/disable ride the same update verb. The composition is rebuilt whole on every provider or binding write and swapped atomically, so enabling, rotating or disabling takes effect without a restart. Every server also re-reads the composition periodically (the 5-minute backstop behind the cross-server invalidation). A re-read whose database read fails — a database outage spanning the tick — is not a composition: the last good composition stays in force, secret resolution keeps working through it, and the re-read retries every 2 s once the database answers; a working composition is never replaced by a refusing one. Only a server that started inside the outage, with no composition read yet, refuses secret resolution by name (the enabled secrets providers cannot be used: …) and heals the same way. The deferral is recorded as the warn:server:secrets_refresh_deferred event once the database answers.
• Test probes with the stored credential exactly as resolution would use it: token auth proves the token via a self-lookup, userpass/approle perform the login, the cloud kinds perform their list-shaped capability probes. The typed outcome rides the response and is recorded as the connection's health — an unreachable vault is a successful call with success: false, the honest replacement for finding out at step time.
• Delete removes a provider — refused by name while scope bindings still reference it, so a delete never silently re-routes bound scopes to the default. Deleting the enabled default means re-creating one or losing vault resolution.

OpsChain is never a vault. The posture rule: the local AGE file store serves the local lane and a server on the SQLite store (both single-node by construction); a server on PostgreSQL resolves only through registered providers and never reads it. On PostgreSQL, no provider registered → every secret-vault:// reference fails by name (no secrets provider is registered …), a step carrying one fails before dispatch, and /api/secrets refuses; providers exist but all are disabled → resolution refuses loudly naming them; the literal reference is never passed through as if it were the value. On the SQLite store the AGE store under the server home resolves references best-effort (an unresolvable one passes through unresolved — a pointer, not a value) and serves /api/secrets, exactly as the local lane's opschain secret … store does.

The server's own integration credentials need no vault. An inline credential supplied to a model provider, identity provider, GitHub or JSM connection, image registry, image build provider or transport provider — and the licence installation credential opschain admin licence register stores — is held encrypted at rest in the database on the record itself, under the server's encryption keys and the key-rotation plane, never written into a secrets provider. Only a *_ref needs a registered server-resolved provider to pin and resolve through. A PostgreSQL server with an empty registry therefore registers every integration; what it cannot do is resolve secret-vault:// references a change uses.

Where references resolve

Routing is unchanged by the locus: the bindings prepass decides the provider per reference exactly as above. (The secret-env://<NAME> scheme never reaches a provider: it resolves only in the local lane, from the process running opschain run, and a server refuses it by name — in a property document at dispatch and on a run input at creation.) Resolution then branches on that provider's resolution:

• server — read, swapped into the step context and masked before dispatch, as today.
• runner — syntax-validated and left in place; the job spec gains an additive secret_refs element, one entry per reference carrying the provider code, the bare reference and the target (which step context pointer, which bound-input variable, or the transport session bundle holds it). The runner resolves each at Prepare through its own configuration, swaps the value in and masks it before any output flows; an unresolvable one is a named infrastructure error naming the reference and the provider code. The spec carries references only, never values, and the step's log says per provider what happened (secrets: provider "platform" - 12 resolved …, secrets: provider "cust-vault" - 7 references deferred to the runner).

A converged document can therefore reference both a platform vault (server) and a customer vault (runner): the server resolves and masks its half, the runner resolves and masks the other, and neither side ever sees the other's values. Each secret_refs entry also carries the reference's current epoch for the run (the declared-write counter of "How steps consume it" below), and the spec ships the provider's rate budget beside them (secret_economy), so the runner resolves under the same economy the server does: once per run per runner from its own run snapshot, re-reading a reference whose epoch moved, metering its reads through the provider's budget and breaker, and reporting counts — never values — up the step's result for the server to write. A command that splices a runner-resolved reference inline (${{ inputs.password }} in run:) is refused by name at the spec build — read it from the step's environment variable (OPSCHAIN_INPUT_<NAME>) or the step context at run time instead.

Placement by declared capability. A runner declares at registration, and re-advertises on every poll, the provider codes its configuration resolves. A step whose references route to a runner provider is placed only on a runner that declared it (secrets_providers joins the placement requirement); with no capable runner in its group the fleet queue reads no runner in group "prod" resolves secrets provider "cust-vault", and an offer that still reaches an undeclaring runner — or a runner predating protocol 8 — fails the step by name rather than handing it a reference it cannot read. The runner's capability shows on opschain admin runners list as secrets_providers.

The runner's configuration is the secrets_providers: section of <runner dir>/config.yaml, one entry per code — kind, address, mount, the auth method with its material (inline, credential_env or credential_file), TLS posture — owned by the customer and never stored in or transmitted by the server (opschain docs runner-executors). The retired OPSCHAIN_VAULT_* runner-host lane refuses the runner's start by name until opschain runner migrate-vault-env has written the section from it, once. Agents resolve runner providers the same way through the SDK, from their own config.yaml, with provider-qualified references.

What a runner provider cannot do. Interactive reads — the governed vault browser and the vault reveal lane authenticate from the server — refuse by name: reads happen only at the runner; the platform holds no credential. The test verb answers the same callout instead of probing: posture is capability-based (which runners declare it), not health-probe-based. No step lease is minted (the server has nothing to mint from); the runner narrows its own credential to a step-scoped child token as before. And a transport provider's credential_ref routed to a runner provider is checked out at the runner: the session bundle is minted with the reference in place and completed there (ssh-ca signs the ephemeral certificate at the runner too).

Server-resolved record fields. The server resolves secret-vault:// references for its own integration credentials, and those cannot be read by a runner, so a write that routes one of them to a runner provider is refused with provider "<code>" resolves at the runner; the server cannot read it: identity providers' client_secret_ref (and bind_password_ref), the GitHub integration's private_key_ref, the JSM integration's api_token_ref, image registries' and image build providers' credential_ref, model providers' credential_ref, a secrets provider's own credential_ref, the licence manager's installation credential, and the /api/secrets owner lane. Transport providers' credential_ref is the one exception (above). In a single-vault estate whose only provider is runner-resolved, keep those credentials inline (encrypted at rest) or point them at a server provider the platform operator runs.

Scope bindings and multi-vault routing

The first enabled provider is base platform and acts as the estate's default resolution source. Enabling a second provider — whatever the kinds — requires the secrets:multi-vault entitlement, as does declaring a scope binding (the typed refusal names the feature).

A binding is one declared scope-path → provider mapping — declared and auditable, never inferred: / binds the estate default, /projects/<code> and /projects/<code>/environments/<code> override it down the node tree, and the deepest covering scope wins (segment boundaries, exactly like the properties overlay). Each binding read carries the bound provider's identity and recorded health, so the posture view answers which parts of the estate resolve against an unhealthy provider, and a resolve verb answers provenance for any node path: which provider governs it and which scope decided — and, given the node that supplied a value (--origin), which provider each reference form resolves through.

Which node's binding serves a reference depends on where the value came from — the layer rule — unless the reference says otherwise:

Form	Routes by	Use it when
secret-vault://<path>/<key> supplied by a database layer — a project, environment or asset document entered in the product, a template layer, a MintModel override lifted from one	the deepest binding covering the node whose document supplied the value	the default: the secret lives where it was written
secret-vault://<path>/<key> supplied by the repository layer (the template's committed properties) or by a per-run override	the deepest binding covering the node resolving it — the step's run target, the refresh task's node, the reveal's owner	the default: one committed path resolves against each estate's own vault
secret-vault://origin@<path>/<key>	the supplying node, regardless of layer	a committed reference that must read the project's vault everywhere
secret-vault://here@<path>/<key>	the resolving node, regardless of layer	a project-entered reference that must follow each environment's vault
secret-vault://<code>@<path>/<key>	the named provider, regardless of everything	the explicit cross-provider read; wins over every rule

The scenario the rule was adopted for: a project keeps its shared secrets in the project's vault (/projects/payments bound to payments), while /projects/payments/environments/prod is bound to prod-vault for its own properties. A reference typed into the project's properties resolves from payments when a change runs under prod — no duplicate in prod-vault, no provider code in the property — while the template's committed secret-vault://db/password resolves from prod-vault under prod and from dev-vault under dev. origin and here are reserved: a provider cannot take either code. The internal ?opschain_level= option another server tags references with is never written by an author and is refused by name.

Three refusals keep routing honest: with several providers enabled and no root binding, default resolution refuses by name rather than guessing which vault governs unbound scopes; a binding naming a disabled provider refuses resolution for its scope loudly rather than silently falling back to the default; and a value whose supplying node no binding covers while no root binding names a default refuses by name, naming the property path and the origin node. There is no fallback between vaults in either direction — a read that cannot be routed fails rather than reading a same-named secret from the wrong vault.

Licence lapse is polite: existing providers and bindings keep resolving — a lapse refuses new enabling writes and binding declarations but retains configuration untouched, and binding deletion stays ungated so a lapsed estate can always unwind. The runner step lease (below) ships ungated: security posture, never metered. See entitlements.

How steps consume it

Just before each step executes, the server resolves the whole-value secret-vault://<path>/<key> references the step carries through the provider the bindings route to, and registers the resolved values with the log masker — logs, events and stored properties only ever show the reference. The reference grammar, secret-typed run inputs and the masking walk-through are opschain learn 3.5. Under the vault traffic economy the platform resolves as the best-behaved client a vault has, in six measures:

One read per run per reference — the run snapshot. A reference is read from its provider once per run, at the first step that needs it, and reused for the run's remaining steps from a run-scoped, in-memory, sealed snapshot in the resolving process — the server's memory for a server provider (every step on every runner sees the same value), each runner's memory for a runner provider (a run spanning four runners forms four snapshots). The snapshot is never persisted and is dropped at run end; a server or runner restart drops it and the run re-resolves — one extra read per path, never a correctness risk. A run therefore sees one consistent snapshot of the vault, and a secret rotated out of band mid-run (a human in the vault UI, another system's rotation) is seen by the next run, not this one. The client's one-minute per-path cache stays as the cross-run floor underneath.

Declared writes move the run on. A step that rotates a secret declares it: secrets_written: [secret-vault://<path>/<key>, …] on the run step (opschain learn 3.5). At that step's successful completion the server bumps the reference's epoch for the run — a row beside the run, shared across replicas, so a failover never resurrects a pre-rotation value — evicts its own snapshot entry, and ships the new epoch to every runner on the run's next specs; a runner whose snapshot entry is older re-reads before the step runs, bypassing its per-path cache too, so a rotation one runner performed is seen by every other runner of the run on its next step. refresh_secrets: true on a step is the blunt override: every reference that step carries is re-read from its provider regardless of epoch. An undeclared rotation is not seen until the next run — stated plainly, by design.

Narrow resolution on the Go lane. A declarative step (run:, a built-in verb, an emit step) binds what it reads explicitly, so only the references its bound inputs and its ${{ properties… }} reads carry are resolved — in the command, the verb's with:, the emit data, the bound inputs and set_properties values; a reference the step never reads stays a reference in its step context, unread and absent from its mask set. The Ruby runner lane (actions.rb#, grafted MintPress trees, bare action names) reads the whole properties document in-action, so it keeps whole-document resolution. A Go-lane step that needs a property secret reads it through ${{ properties.<key>.<path> }} or binds it as a secret-typed input.

The vault kinds' in-runner lane and the run-scoped lease. For the vault kinds a server provider additionally has the server project its connection facts into the action step's runner as the OPSCHAIN_VAULT_* environment — the variable set opschain-core reads during action setup to resolve secret-vault:// references inside the step itself (a runner provider's facts are projected by the runner from its own configuration instead). That in-runner resolution lane is the Ruby runner lane, so it is only applicable to existing MintPress customers and those using the MintPress SDK or the packs:mintpress feature; every other step gets its properties resolved before dispatch, masked in logs. The cloud kinds resolve server-side only and project no runner environment — each pack page records that gap precisely. On that projected lane the estate credential never rides the job spec: the server mints a response-wrapped child token from the step's bound provider once per run, runner and provider (runner protocol v9+; TTL bounded to the run's timeout plus slack, non-renewable, metadata carrying the run and runner identity — never values) and ships it as the spec's additive vault_lease on that runner's first step of the run; the runner unwraps it, holds it for the run and bridges it into OPSCHAIN_VAULT_TOKEN just-in-time. Later steps of the run to the same runner carry the held marker instead of a wrapping token and the runner answers them from the token it holds; a different runner gets its own mint (a lease never propagates between runners — the wrapping token is single-use by design); a runner that restarts mid-run has a new session, so its next step carries a fresh mint and the old lease is revoked by accessor; at run end the server revokes every lease of the run by accessor, expiry the backstop. The wrapping token is single-use with a short TTL, so one that expired or was already unwrapped refuses loudly at the runner — the interception-evidence property. Path scoping rides the standard token-role mechanism (a mount-scoped ACL policy and token role, provisioned idempotently); an estate whose server credential may not write policies degrades one tier (the child carries the parent's own policies), and one refusing child minting entirely degrades to the credential-env projection — both degrades named in the step's log, never silent. Sessions predating protocol v9 keep the per-step mint; sessions predating the lease protocol (v6), and in-process dispatch, keep the credential env projection: the payload is additive both ways. The step-scoped child token the runner mints from its own credential on the env lane follows the same per-run rule.

The rate budget, 429 honoured, and the breaker. Every real call to a provider — a read on a cache miss, a token mint — passes the provider's rate budget (rate_limit_reads_per_second, rate_limit_mints_per_minute; the estate default secrets.default_rate_limit under a lane left at 0; -1 on a lane lifts it at this provider whatever the default; unset everywhere = unbounded): a call past the budget waits for its slot in a bounded queue, the step's log saying waiting for vault budget (provider "<code>"), and a slot further away than the bound refuses the step by name rather than queueing it forever. A 429 (or a cloud kind's throttling response) is honoured with its Retry-After and retried before the retry budget is spent — never counted as a failure on the first hit. A provider in sustained refusal (five consecutive 429 or 5xx answers) trips its circuit breaker: for a one-minute cool-down steps fail fast by name (secrets provider "<code>": the vault circuit breaker is open …) instead of hammering the vault, the provider's dashboard shows the breaker open (the breaker_open gauge), and the first successful call after the cool-down closes it.

Counters the vault administrator can be shown. Every measure is counted on the provider usage substrate ("Usage telemetry" below) and summed into one line of the run's evidence (info:run:vault_traffic): vault traffic: provider "platform" 50 reads (152 from the run snapshot), 1 lease; provider "cust-vault" 7 reads at the runner.

Interactive reads are different from platform resolution: the decrypt/reveal lane authenticates to the vault as the asking user, with each user's own stored vault credential and the vault's own ACLs deciding what they may read — no stored personal credential, no read. The verbs, the step-up floor on decrypt-for-display and the audit trail are in opschain learn 3.5.

Writes are explicitly addressed, never binding-routed. A store (vault store / vault store-file, or the store API verbs) with an unqualified secret-vault://<path>/<key> reference lands in the platform's default provider, as always. A provider-qualified reference (secret-vault://<code>@<path>/<key>) writes through the named provider instead, authenticated as the asking user with their own stored credential — the same identity rule as interactive reads. An unknown or disabled provider code refuses by name, a caller without a personal credential for the named provider gets the typed credential refusal, and a qualified write is never silently redirected to a different vault. Scope bindings route step reads; they never redirect a write.

Usage telemetry

Every read a secrets provider serves is counted on the provider usage substrate (opschain docs provider-usage): reads (a read against the provider, with its latency; a failed read is an error), cache_hits (a read the client's one-minute per-path cache answered instead), snapshot_hits (a reference the run snapshot answered without a read), token_mints (a run-scoped vault child token minted at dispatch; a refused mint is an error), token_revokes (a lease revoked by accessor at run end or on a runner's restart), queue_waits (a call that waited for the provider's rate budget), rate_limited (a 429 or throttling answer, honoured and retried; an error) and breaker_trips (the breaker opened; an error), plus the breaker_open gauge (1 while the breaker is open) the posture page and the providers overview show — each on the project of the dispatching run where there is one. A runner provider's counts are reported by the runner on each step's result and written by the server; values never travel. opschain admin providers usage --kind secrets --code <code> reads them at any grain; the provider's record opens its operational dashboard; each run's evidence carries the info:run:vault_traffic line.

Settings and configuration

No config.yaml key tunes this component at runtime. The retired vault.* provider keys (and the server-process OPSCHAIN_VAULT_* environment) survive only as the upgrade seed's input: they are consumed once at boot to write one enabled provider (recommended code vault), after which a config.yaml still carrying provider keys refuses boot by name — see opschain docs config. The seed never connects to the vault; the operator's first action is the test verb. Per-provider knobs (cipher_stacking, password_length, TLS posture) live on the record itself.

Governance and audit

Every verb sits behind the /admin updatable determination (permission denied on deny), over the v2 user-token lane — the caller's own opschain login session, with the admin-token fallbacks the CLI manual records. Compositions without the secrets-provider plane answer not-found, the nil-domain convention.

Events (all catalogued on opschain docs events):

• api:secrets_providers:create / update / destroy — connection lifecycle; connection facts only, never a credential, a rotation carrying credential_rotated: true.
• api:secrets_providers:test — a probe ran; the provider's code and the success flag.
• api:secrets_provider_bindings:create / update / destroy — a scope binding declared, re-pointed or removed; the scope path and the bound provider's code.

Entitlements, in one line each: the component itself carries no gate; the cloud kinds gate on packs:secrets-aws / packs:secrets-azure (create/enable only, polite lapse); multi-vault — the second enabled provider and every binding declaration — gates on secrets:multi-vault (polite lapse, binding deletes ungated). All three are entitlements territory.

Related pages

• opschain docs cli/admin-secrets-providers — the command family (list, show, create, update, delete, test), verb by verb.
• opschain docs runner-executors — the runner's secrets_providers section and the secret_refs spec element; opschain docs cli/runner-migrate-vault-env — the one-off migration.
• opschain docs cli/admin-secrets-bindings — binding semantics (list, set, delete, resolve).
• opschain docs getting-started/secrets-providers — the operator walkthrough; opschain learn 3.5 — writing and using secrets.
• opschain docs packs-secrets-aws / opschain docs packs-secrets-azure — the cloud kinds' configuration detail.
• opschain docs entitlements — the gates and the lapse posture.
• The API service is the v2 SecretsProviderService (opschain docs api-v2).