System information
Understanding the system information screen
Select Administration in the navigation bar to open the administration screen. Its first section, System information, shows the state of the OpsChain instance you are connected to, and refreshes every ten seconds while it is open. It is split into tabs:
| Tab | Shows |
|---|---|
| Summary | The headline figures from the other tabs on one page. |
| Nodes | Each Kubernetes node in the cluster. See nodes. |
| Queues | The job queues the OpsChain workers process. See queues. |
| Pods | Every pod in the OpsChain namespace, with their logs. See pods. |
| Deployments | Every Kubernetes deployment making up the instance, which you can restart or scale. See deployments. |
| Background tasks | Every background task running or queued against a node. See background tasks. |
| Database | The database clusters of every site, and the database the API is using. See database. |
The Summary tab shows:
- Nodes - Total nodes, with the CPU and memory use of the first node.
- Queues and jobs - Total queues, and the number of jobs running, pending, scheduled, failed and finished across every queue, with their total.
- Pods - Total pods, and each pod with its state.
- Database info - the same database details as the Database tab.
Access is controlled by the /admin resource path.
The Nodes and Pods tabs describe the site whose OpsChain API you are connected to. In a high availability deployment each site runs on its own Kubernetes cluster, so connecting to another site's API shows that site's nodes and pods instead. The queues are held in the database the sites share, so the Queues tab is the same from every site.
Nodes
The Nodes tab describes the Kubernetes nodes - the servers - that the cluster running OpsChain is made of. For each node it shows its CPU and memory use, its status and versions, the conditions Kubernetes reports on it, and the container images it holds.
The CPU and memory figures come from the Kubernetes metrics server. If they cannot be fetched, the tab says so and shows the rest. K3s installs the metrics server by default; on another Kubernetes distribution, see metrics server.
Queues
OpsChain's workers take their background work from job queues. The Queues tab shows how many jobs of each kind are running, pending, scheduled, failed and finished in each queue, so you can see whether work is waiting, failing or not being picked up at all. See stopping and starting the workers for what the workers are.
The figures across the top give Total queues and the job counts summed over every queue. Below them, the table has a row for each kind of job in each queue:
| Column | Description |
|---|---|
| Queue | The queue the jobs are in. |
| Job class | The kind of job. |
| Running | How many of these jobs are running now. |
| Failed | How many have failed. |
| Finished | How many have finished. |
| Pending | How many are waiting to be picked up. |
| Scheduled | How many are scheduled to run later. |
| Total | The total number of these jobs. |
| Next run | When the next of these jobs is due to run. |
| Last run at | When one of these jobs last ran. |
| Highest error count | The most times any one of these jobs has failed and been retried. |
Use the Queue filter to show only the jobs in the selected queues.
A Pending count that keeps growing while Running stays at zero means no workers are taking jobs from the queue - check the opschain-api-worker row on the deployments tab.