Database
The Database tab, in the System information section of the administration screen, describes the database behind your OpsChain instance: the database clusters of every site in the deployment, and the state of the database the API you are connected to is using.
Topology
OpsChain can be deployed as a single site or as several sites that share their data — see the high availability guide for the topologies this supports. A site is one OpsChain deployment: an API, its workers, and its own database cluster. One site's database cluster holds the data and the others hold a copy of it, which is how the sites stay in step.
The Topology section, at the top of the tab, shows each site's database cluster as a card. The one holding the data the others copy from comes first, marked Primary. A site appears here once something in it has reported in — either its API starting up, or its workers picking up work — so a site added to the deployment shows up on its own within a minute or two, with nothing to register by hand.
When more than one site is listed, each card has a Cordon button that stops new work from starting at that site, and a cordoned site's card has Uncordon instead — see cordoning a site. The button is disabled for users without permission to cordon a site.
A non-primary site whose API has never started — one deployed in stopped mode, holding only a copy of the data — is still listed, using the name its database cluster is known by, but only its replication state can be shown. Everything the site would have to report about itself reads as -.
| Field | Description |
|---|---|
| Role | Whether this site's database cluster was deployed to hold the data the others copy from (primary), or to hold a copy of it (replica). |
| Replication | How this site's database cluster is keeping up, shown as a labelled state. See replication states below. |
| Replication lag | How far behind this site's database cluster is, in seconds. A write made on the primary takes this long to show up here, so a site that is keeping up is normally behind by a fraction of a second. The primary has nothing to be behind, so it reads as -. |
| Workers active? | Whether workers at this site are currently picking work up. They report in every minute and are treated as stopped once five minutes pass without one, so this can take up to five minutes to catch up after workers stop. |
| Last seen | When a worker at this site last reported in. This tracks the workers only — not the API, and not the database. |
| API started | When this site's API last started. Useful for confirming that a site restarted when you expected it to. |
| Cordoned | Shown only for a cordoned site: when it was cordoned and by whom. The card's title also shows a Cordoned badge. |
| Reason | Shown only for a cordoned site: the reason given when it was cordoned, or - if none was given. |
| Still running here | Shown only for a cordoned site: how many changes, agents and background tasks started there before it was cordoned and are still running. Running agents stay at the site until they are stopped. |
Replication states
The Replication field describes how each site's database cluster is keeping up. Hover over the value to see the same explanation in the screen.
| State | Meaning |
|---|---|
| Primary | This site's database cluster holds the data the other sites copy from. There is nothing for it to be behind. |
| Up to date | This site's database cluster is up to date. One that is keeping up is normally behind by milliseconds, so a small lag alongside this state is expected. |
| Behind | This site's database cluster is behind and is still catching up. A brief spell here is normal after a burst of activity; a site that stays here needs investigating. |
| Disconnected | A replication slot for this site exists but nothing is attached to it, so its database cluster is not being updated. It will start catching up once it reconnects. |
| Needs rebuilding | This site's database cluster has fallen too far behind to catch up on its own, because the changes it still needs are no longer available. It has to be rebuilt — see recovering a replica cluster. |
The replication state is only known for clusters that the primary database can see, so it reads as - for a site that replicates from another site rather than from the primary.
OpsChain records an event whenever a site's database cluster changes role, so a failover is visible after the fact rather than only while it is happening. Look for info:database_cluster:role_changed in the audit history to see which site took over and when. A site joining the deployment for the first time is recorded as info:database_cluster:registered.
Comparing deployment configuration
Each site records the deployment-managed settings that were applied to it at install or upgrade time — values such as service hostnames and ports, the Kubernetes namespace and the time zone. These are the settings managed by the deployment; they are owned by each site's values.yaml file rather than edited through the GUI, so they can differ from one site to another.
The gear icon on the right of a site's card opens the configuration recorded for it. From there you can:
- Compare two sites. When you operate more than one site you can compare their configuration side by side to spot the difference between them.
- Compare a site against itself over time. OpsChain keeps a history of each site's recorded configuration, so you can see what changed between deploys and when. This is useful when troubleshooting an issue that began after an upgrade.
The name of the site you are currently connected to is shown in the version info dialog.
Database settings
The Database settings section shows how the API you are connected to reaches its database. In a high availability deployment every site's API connects to the same data, so these values describe the connection rather than anything site-specific.
| Field | Description |
|---|---|
| Host | The hostname the API connects to. |
| Port | The port the API connects on. |
| Database name | The name of the database OpsChain stores its data in. |
Database info
The Database info section reports the state of that database as it stands now.
| Field | Description |
|---|---|
| Connections | The maximum number of connections the database is configured to accept, with the number currently active and idle listed beneath it. A count of active connections approaching the maximum is what to look for when the API starts refusing work. |
| Database size | The total size of the OpsChain database on disk. |
| Table | The five largest tables and their sizes, so a database growing faster than expected can be traced to what is growing. |
| Uptime | How long the database has been running since it last started. A short uptime you did not expect means the database restarted. |
| Version | The PostgreSQL version the database is running. |
Query planner statistics
The database chooses how to run every query from statistics it gathers about each table, so a table whose statistics have gone stale can make the queries that read it slower. The Query planner statistics section lists the tables with the stalest statistics first, so the tables the planner knows least about lead the list.
| Field | Description |
|---|---|
| Table | The name of the table. |
| Rows | The number of rows the planner believes the table holds. Shown as — for a table that has never been analysed, which has no estimate. |
| Modifications since analyze | How many rows have been inserted, updated or deleted since the table was last analysed. A large number against a table the planner last looked at long ago is what to look for. |
| Last analyzed | When the table was last analysed, whether by the database itself or by a cleanup job. Never means its statistics have never been gathered. |
To bring them up to date, schedule a cleanup job with Refresh database statistics enabled — see refreshing database statistics.