Maintenance windows

maintenance

A maintenance window quiesces exactly one node's subtree — a project, environment or asset path — while declared maintenance work runs. Under a window, non-allowlisted run creation targeting the subtree is refused with a named 422, and trigger firings whose action targets the subtree park as deferred — never lost — until the window ends. Everything else is untouched: siblings proceed normally, and in-flight work already running when the window starts finishes on its own terms (the gate runs at creation only).

Its neighbours: the trigger engine (opschain docs triggers) both defers firings under windows and drives the scheduled-window lifecycle; the events stream (opschain docs events) carries the window's lifecycle events, and ending a window is itself an event the engine sweeps to release the parked work. This is node-scoped maintenance — distinct from the estate-wide postures (opschain docs platform-halt).

Surfaces

Surface	Does
GET /api/projects/…<node path>/maintenance	The node's window status (data: null, meta.active: false when none).
POST …<node path>/maintenance	Start a manual window now (201).
POST …<node path>/maintenance/end	End the active window (200 with the ended window).
POST …<node path>/maintenance/schedule	Create a scheduled window (201).
POST …<node path>/maintenance/cancel	Remove a scheduled window that has not started (window_id required).
POST …<node path>/maintenance/extend	Lengthen the active window's end.
GET /api/maintenance	Estate listing: every active and scheduled window, readable by any authenticated caller (when a window gates a change's start, every reader of the change may see which); ?ended=true adds ended ones, ?since=<RFC3339> bounds them.
GET /api/maintenance/{window_id}/deferred_firings	The window's parked firings, original event order.
POST /api/projects/{p}/triggers/{id}/firings/{firing_id}/discard	Discard one parked firing before release.

The CLI family mirrors these one-for-one: opschain maintenance start, schedule, status, extend, end, cancel and list (opschain docs cli/maintenance), plus opschain triggers discard (opschain docs cli/triggers-discard) for the firing discard.

The window, attribute by attribute

One maintenance_window resource (type maintenance_window, self link <node path>/maintenance):

Attribute	Meaning
active	The window is quiescing right now (started_at set, ended_at not).
scheduled	The window awaits its auto-start (started_at and ended_at both null).
reason	Why the node is under maintenance — required, echoed in every denial.
until	A manual window's declared end intent. Advisory: the window never auto-closes on it, but it bounds deferral expiry at release.
scheduled_start	A scheduled window's declared start (from); the engine activates the window when it arrives.
hard_end	A scheduled window's enforced end (to); the engine ends (or auto-extends) the window at it.
auto_extend_secs	Per-increment auto-extension applied at the hard end while declared work is unfinished.
max_extension_secs	The fail-open cap on total auto-extension (defaults to 7200 — two hours — when auto-extension is asked for without one).
auto_end	End the window as soon as its stamped work — armed by the first stamped run — has all finished, whatever the outcome.
runs	The workflow codes the engine starts at auto-start, as the scheduling operator. They are folded into allow.workflows, so the window's own work is admitted and stamped allowed.
allow	The declared maintenance work: {workflows, actions, users} string arrays. Any one list matching a create (workflow code, change action, requesting username) admits it.
started_by / started_at	Who opened the window and when (started_at null while scheduled).
ended_by / ended_at	Who closed it and when (ended_by null on an engine-driven end).
deferred_count	The live count of firings parked by this window (status deferred) — the parked-work badge.

Nodes covered by an active window also carry a maintenance badge in their own serialization: reason, until and window_path (the quiesced node, so a badge inherited from an ancestor window points at the window itself). A scheduled window starting within the server's badge horizon (api.maintenance_badge_horizon_minutes, default 60) badges its covered nodes the same way with scheduled: true and starts_at (its declared start); until then carries the scheduled hard end. The badge is display only — nothing is enforced until the engine activates the window.

Lifecycle

One not-ended window per node — active or scheduled — enforced by a partial unique index, so a racing second start/schedule loses cleanly: A maintenance window is already active on <path> or A maintenance window is already scheduled on <path>.

• Manual: start opens immediately; only end closes it (until is intent, nothing enforces it). Ending with none active answers No maintenance window is active on <path>.
• Scheduled: the trigger engine's 2-second tick owns every hands-off transition, each one a guarded UPDATE so HA peers race safely and exactly one engine performs it. A due window CAS-activates at scheduled_start (emitting api:nodes:maintenance_started so gating, deferral, badges and triggers engage exactly as a manual start — nodes soft-deleted in the meantime are skipped), then creates its declared runs as the scheduling operator: human lineage, stamped allowed. At the hard end the engine either auto-extends (auto-extension set, stamped work still unfinished, cap not reached), ends loudly at the cap (fail-open: ending never kills in-flight work, it only resumes deferred automation), or just ends; an auto_end window ends as soon as its stamped work has all finished.
• Cancel removes a scheduled window that has not started; the delete is compare-and-set on started_at IS NULL, so a cancel racing the activation loses: Window <id> has already started — end it with the maintenance end command instead of cancelling.
• Extend lengthens the active window — the hard_end of a scheduled window, the advisory until of a manual one — taking exactly one of until (a new end) or by_secs (seconds to add). Lengthen only: Extending can only lengthen the window — to finish earlier, end the window instead. The update is compare-and-set on the end it read, so a concurrent auto-extension or manual extend answers The maintenance window on <path> changed while extending it — check its current end and retry instead of silently shortening the other's result.

What a window defers or blocks

The gate runs at run creation — every door: user/API change and workflow-run creation, recommendations, trigger firings, and nested runs spawned by workflow steps. Scope is strict: a window covers exactly its own node and the subtree below it, on segment boundaries; a scheduled window quiesces nothing until the engine activates it. Under one or more covering windows:

1. Trigger lineage never creates. The engine consults the gate before any create: a firing whose resolved action target sits under an active window parks as deferred, with deferred_window_id recording the window and the reason deferred: <path> is under maintenance: <reason> in its error column, and an info:triggers:deferred hop on the matched event's chain. Trigger lineage is never allowlisted and never overrides.
2. Allowlisted work is admitted when every covering window's allow admits it (any one list — workflow, action or username — per window), stamped allowed on the created run.
3. A workflow step's spawn inherits its parent run's stamp — the allowance rides the human lineage as one authorised operation.
4. override_maintenance on the create payload admits, stamped overridden, only for principals holding update on every covering window node's …/maintenance rule path — audited as audit:maintenance:override.
5. Everything else answers the named 422 carrying the innermost window's reason: <path> is under maintenance: <reason>.

Release, expiry and discard

Ending a window emits api:nodes:maintenance_ended; when the trigger engine sweeps that event (or boots onto a window that ended while nothing was watching), every still-deferred firing of the window is re-evaluated in original event order through the same rails a fresh sweep applies: the trigger must still exist, be enabled and still match the deferred event, then throttle, depth cap, templates, a possible re-defer under another covering window, RBAC and the create. Exactly-once across HA engines rides a guarded deferred→pending CAS per firing.

• A deferral released after the window's declared until expires instead of firing into a world nobody is watching: it settles suppressed with deferral expired: the maintenance window on <path> was declared until <time> and ended after it — nothing is created, and a warn:triggers:deferral_expired event lands on the window's node.
• A trigger disabled (or no longer matching) while its firing was parked settles the firing suppressed, visibly, with a named reason.
• Discard prunes one parked firing before release: only a deferred firing discards (Firing <id> is not deferred — only a deferred firing can be discarded); it settles suppressed with discarded before release by <user>, keeps deferred_window_id as history, and emits the settled hop. A discard losing the status CAS to a concurrent release answers Firing <id> is no longer deferred — a concurrent release claimed it first.

Levers, and who owns them

Lever	Authority	Effect
Start/end/schedule/cancel/extend	Update on the node's …/maintenance rule path (API verbs / opschain maintenance *); read on it gates the per-node status and the deferred-firings drill-down, while the estate listing is open to any authenticated caller. 404-on-deny.	Opens, closes and reshapes the window.
allow lists and runs	Set on the start/schedule payload	What the gate admits as the window's declared work.
until / from–to / auto_extend_secs / max_extension_secs / auto_end	Set on the start/schedule payload; extend moves the end later	The window's timing and the engine's hands-off decisions.
override_maintenance on a create payload	The caller, per request — honoured only with update on every covering window's …/maintenance rule path	Admits one create through the gate, stamped overridden and audited.
Discard of a parked firing	Read on the trigger's rule path plus update on the deferring window node's …/maintenance rule path	Settles one deferred firing suppressed before release.
Licence feature maintenance	The installed licence (opschain docs entitlements)	Start and schedule refuse with the entitlement's named 422 when the feature is off (Basic tier without the feature).

No config.yaml key tunes this component; the lifecycle cadence is the trigger engine's built-in 2-second tick, and the default auto-extension cap (two hours) is built in.

Events vocabulary

All are ordinary triggerable events, visible in opschain events:

• api:nodes:maintenance_started — a window opened (manual start, or the engine's auto-start with "scheduled": true); on the window's node.
• api:nodes:maintenance_ended — a window closed (manual end, hard end, auto-end or the extension cap; the engine's end carries a message naming which). Sweeping this event is what releases the deferred firings.
• api:nodes:maintenance_scheduled — a scheduled window was created.
• api:nodes:maintenance_extended — the end moved later (manual extend, or an auto-extension carrying auto_extended_secs).
• audit:maintenance:override — the escape hatch was used: who, which window, what was created.
• warn:maintenance:auto_extend_cap — an auto-extending window ended at its extension cap with work still running (fail-open, loud).
• warn:maintenance:declared_run_failed — a declared workflow could not start at auto-start (including a licence work-stop); the window continues, the failure is loud.
• info:triggers:deferred, warn:triggers:deferral_expired and the api:triggers:fired settle hops — the trigger engine's side, see opschain docs triggers.

Failure modes

• A second window on the same node is refused at start/schedule time with the named 422; the partial unique index is the race-safe backstop.
• A declared run that cannot start never stalls the window silently — the failure lands as warn:maintenance:declared_run_failed on the node and the window proceeds.
• The extension cap ends an auto-extending window with work still running: in-flight work keeps running, deferred automation resumes, and warn:maintenance:auto_extend_cap says so.
• A window that ended during a whole-fleet restart is not lost: each engine sweeps once at start for deferred firings whose window ended while no engine was watching, and releases them (the per-firing CAS keeps this safe beside a peer's concurrent boot or live sweep).
• A parked firing whose matched event was retention-pruned settles suppressed (released from maintenance, but the matched event no longer exists — nothing is created); no chain hop can be emitted without the event's spine.
• A hand-emitted maintenance_ended event for a window that is still active (or never existed) releases nothing — only the real end transition releases.

Related pages

• opschain docs cli/maintenance — the CLI family, verb by verb.
• opschain docs triggers — the engine that defers, releases and runs the scheduled lifecycle.
• opschain docs events — the event stream the lifecycle rides.
• opschain docs platform-halt — the estate-wide postures, which are not maintenance windows.