Introduction
OpsChain requires TLS certificates for its internal communication with its services, such as the image registry and the build service. It can also be configured to use TLS/HTTPS connectivity for its API and web UI, ensuring that all communication is encrypted and secure.
This guide will walk you through the necessary information to understand how to configure TLS/HTTPS connectivity for OpsChain and the options available to you.
Networking
OpsChain uses Kong Ingress proxy to route traffic to its services, this means that the hostname on which OpsChain and its services are accessible defaults to the hostname of the VM it is installed on. It is your responsibility to ensure that this server's hostname is reachable from your machine/network by creating the appropriate DNS entries for it.
If you are using the OpsChain vault as the global default, you should add two DNS entries pointing to the server's hostname, one with the API hostname and the other with the secret vault's external hostname.
Port mapping
All services sit behind the Kong Ingress proxy, which listens on port 3000 for HTTP and port 3443 for HTTPS connections. The requests are then forwarded to the appropriate services based on the request hostname.
OpsChain services listen on the following ports by default:
| Service | Port(s) |
|---|---|
| API and UI | 3000 for HTTP, 3443 for HTTPS |
| Build service | 50000 |
| Database | 5432 |
| Image registry | 8000 |
| LDAP | 389 |
| Log aggregator | 24220, 24224 |
| Secret vault | 8200, 8201 |
These ports are only accessible internally from the Kubernetes cluster and the machine it is hosted on. Unless you expose them via an external service, they won't be accessible from outside the cluster. See the Kubernetes topology reference for the services these ports belong to.
Additionally, when enabling external services for the secret vault or the database (for high availability setups), these ports will need to be opened on the firewall to allow incoming connections from the other instances.
Disabling the insecure HTTP listener
You can disable the Kong Ingress proxy's insecure HTTP listener on port 3000 by setting kong.proxy.http.enabled to false in your values.yaml file.
kong:
proxy:
http:
enabled: false
Only disable the insecure HTTP listener after ensuring you can reach the API, UI and secret vault via HTTPS.
Certificates
OpsChain requires the following certificates for its services to function correctly:
- Image registry certificate:
This certificate is for the internal communication between the image registry and the API, using the hostname configured via
env.OPSCHAIN_IMAGE_REGISTRY_HOST. - Build service certificate:
This certificate is for the internal communication between the build service and the API. The build service hostname is
opschain-build-serviceand cannot be changed.
If you use the OpsChain vault as the global default, then you will also need the following certificates:
- Secret vault internal certificate:
This certificate is for the internal communication between the secret vault and the API. The secret vault internal hostname is
opschain-secret-vault-0.opschain-secret-vault-internaland cannot be changed. - Secret vault external certificate: This certificate is for accessing the secret vault's UI from the client machines.
*.local.gd does not reliably resolve to 127.0.0.1The provided self-signed certificates use *.local.gd hostnames on the assumption that this domain's DNS automatically resolves any subdomain to 127.0.0.1. This is not guaranteed — local.gd's DNS is controlled by a third party outside LimePoint, and it does not currently resolve opschain-image-registry.local.gd (or any other *.local.gd subdomain) to anything at all. Do not rely on it. Instead, add an explicit entry for each *.local.gd hostname you use to the server's hosts file:
echo "127.0.0.1 opschain-image-registry.local.gd" >> /etc/hosts
If your /etc/hosts file is managed by a configuration management tool such as Ansible or Chef, ensure that it is updated to include the entry. See image registry hostname resolution for the equivalent guidance when using a hostname of your own choosing instead.
Optional certificates
Optionally, you can also use HTTPS for OpsChain's API and web UI, which will require an additional API certificate, used exclusively for external access to the API and UI.
You can first install OpsChain without HTTPS connectivity for the API, and then configure it to use HTTPS later.
Trusting additional certificate authorities
This section is only relevant if you are using the OpsChain secret vault as the global default.
Two of the vault's certificates are involved, and they do not have to come from the same authority:
- the internal certificate, presented on the connection OpsChain opens to the vault inside the cluster
- the external certificate, presented by the ingress at
global.secretVaultExternalHostName. The vault runs as several replicas, and a request that lands on a standby is redirected there, so OpsChain reaches this certificate during normal operation, not only from a browser
If those two come from different authorities, OpsChain must trust both. The API contacts the vault as it starts, so an authority it does not trust stops the API from starting rather than failing a single request. See secret vault requests fail with certificate verify failed if that has already happened.
OpsChain keeps the certificate authorities you trust in its trust store, and every OpsChain workload uses them - the connections the API makes to the secret vault included. Adding an authority to the trust store is all that is required; no vault setting needs to name it.
Supplying an authority at install or upgrade
Set trustStore.certificates in your values.yaml file, keyed by the filename to hold each authority under:
trustStore:
certificates:
corp-root-ca.pem: |
-----BEGIN CERTIFICATE-----
MIIDDzCCAfegAwIBAgIUccZDqIbWK49Sr9wEovijlne0XXowDQYJKoZIhvcNAQEL
-----END CERTIFICATE-----
To read an authority from a file rather than pasting it, add --set-file to the installation command:
--set-file trustStore.certificates.corp-root-ca\.pem=/path/to/corp-root-ca.pem
Supplied this way, the authority is in place before the OpsChain API pod is created, so it is trusted the first time the API starts. Use this route for an authority the API needs in order to start at all.
Certificates must be PEM encoded. A certificate exported from Windows is DER encoded by default, so convert it first:
openssl x509 -inform der -in corp-root-ca.crt -out corp-root-ca.pem
Supply your values with -f values.yaml on every upgrade. helm upgrade reuses the previous release's values when it is given none, so an authority removed from values.yaml stays in place until the file is supplied again.
Uploading an authority to a running instance
Upload the certificate in the CA certificates section of the system configuration page. Certificates uploaded there and certificates supplied in values.yaml are held alongside each other, and an upgrade leaves the uploaded ones untouched.
Uploading needs a running API, so it cannot be used to trust an authority the API itself needs in order to start. Supply that one in values.yaml.
Where the trust store is mounted
The trust store is the opschain-trust-store config map, mounted at /etc/pki/ca-trust/source/anchors/ in the OpsChain pods that connect to other services on your behalf. Each of those pods rebuilds its trusted certificate bundle whenever the config map changes, so an authority added to a running instance is picked up without restarting anything - allow a minute or two for Kubernetes to publish the change into each pod.
The same certificates are built into the images that run your steps, so your actions trust the authorities the rest of OpsChain does. See step runner for how that image is assembled.
Hostname configuration
You can configure the hostnames that your certificates use (or should be issued for) by each service in your values.yaml file. After configuring your hostnames, follow one of the available options guide to configure your certificates.
When using your own certificates or provided self-signed certificates, it is your responsibility to ensure that the certificates include a DNS subjectAlternativeName that matches the hostnames you defined in your values.yaml file.
API and web UI hostname
By default, the OpsChain API will use the opschain-api hostname, which points to its Kubernetes service, meaning it will be accessible at your server's external IP address/hostname. You can change the hostname used by the API to something more friendly by setting the api.hostName and the env.OPSCHAIN_API_HOST_NAME values in your values.yaml file.
api:
hostName: "opschain.my-company.com"
env:
OPSCHAIN_API_HOST_NAME: "opschain.my-company.com"
The OpsChain API and UI always share the same hostname.
You must also configure the env.OPSCHAIN_GUI_BASE_URL setting in your values.yaml file to ensure that links within notifications are valid. For example:
env:
OPSCHAIN_GUI_BASE_URL: "https://opschain.my-company.com"
If the API and UI are accessible via a non-standard port, like 443 for HTTPS and 80 for HTTP, you must include the port in the URL. For example: https://opschain.my-company.com:3443. You should also include the scheme in the URL, either https or http, depending on the protocol you opted to use.
Build service hostname
The default build service hostname is opschain-build-service and cannot be changed.
Image registry hostname
Unlike the API and web UI hostname, the image registry hostname is not set to a working value out of the box — env.OPSCHAIN_IMAGE_REGISTRY_HOST and the related trow settings are blank by default. You must explicitly set the trow.trow.domain, trow.ingress.hosts[0].host, trow.ingress.tls[0].hosts[0] and the env.OPSCHAIN_IMAGE_REGISTRY_HOST values in your values.yaml file, either to opschain-image-registry.local.gd (the hostname used by the provided self-signed certificates) or to a hostname of your own choosing. For example:
trow:
trow:
domain: "image-registry.my-company.com"
ingress:
hosts:
- paths: [ "/" ]
host: "image-registry.my-company.com"
tls:
- secretName: "opschain-image-registry-cert"
hosts:
- "image-registry.my-company.com"
env:
OPSCHAIN_IMAGE_REGISTRY_HOST: "image-registry.my-company.com"
If env.OPSCHAIN_IMAGE_REGISTRY_HOST is left unset, the opschain-image-registry-cert certificate will fail to render and the installation will fail.
Once set, you must also ensure the internal image registry hostname resolves to the Kubernetes node's IP address so that OpsChain can access it. It's recommended you do this mapping via your company's DNS server, but you can also achieve this by adding the following entry to the server's hosts file:
echo "127.0.0.1 image-registry.my-company.com" >> /etc/hosts
If your /etc/hosts file is managed by a configuration management tool such as Ansible or Chef, ensure that it is updated to include the entry.
The hostname you use may vary if you have used a different domain name for the internal image registry in your values.yaml file.
Secret vault hostname
This section is only relevant if you are using the OpsChain secret vault as the global default.
The OpsChain vault has two hostnames:
- Internal hostname:
opschain-secret-vault.opschain-secret-vault-internal- This hostname cannot be changed. - External hostname: the hostname you'll use to access the secret vault's UI.
The external hostname must be configured in your values.yaml file by setting the global.secretVaultExternalHostName value:
global:
secretVaultExternalHostName: "vault.my-company.com"
The OpsChain secret vault uses the same ingress as the API, ensure that the host name you provide for the secret vault is different than the API host name.
For the clients that need to access the secret vault UI, you must configure your DNS to map the secret vault hostname to the server where OpsChain is installed.
The secret vault runs as several replicas. A request from OpsChain — the API, workers, and runner pods — can land on a standby replica, which redirects the caller to the active replica at the secret vault external hostname. For that redirect to succeed, the external hostname must resolve from inside the Kubernetes cluster, not only from client machines.
You can test this by running the following command with your own vault hostname on a running OpsChain instance:
kubectl -n ${KUBERNETES_NAMESPACE} exec -it deploy/opschain-api-worker -- getent hosts vault.my-company.com
If the command returns a result, the hostname resolves from within the cluster. If it does not, you need to configure your DNS to resolve the hostname either by adding it to your organisation's DNS, or, on K3s, adding it to CoreDNS with a coredns-custom ConfigMap:
apiVersion: v1
kind: ConfigMap
metadata:
name: coredns-custom
namespace: kube-system
data:
opschain-vault.server: |
vault.my-company.com {
hosts {
192.168.0.10 vault.my-company.com
fallthrough
}
}
Apply it and restart CoreDNS:
kubectl apply -f coredns-custom.yaml
kubectl -n kube-system rollout restart deployment coredns
In a high availability setup, each cluster's CoreDNS must resolve the external hostname of every cluster's secret vault, since the active vault can move between clusters.
Available options
There are a few options available for configuring TLS/HTTPS connectivity for OpsChain, each with their own advantages and disadvantages:
- Using
cert-managerto automatically manage certificates for you. This is the recommended approach if you want to quickly setup your instance and don't want to manage certificates manually. - Using the provided self-signed certificates. This option is useful if you want to trial OpsChain without
cert-managerand don't have your own certificates yet. - Bringing your own certificates. This option is the most flexible if you are running in an enterprise environment and can only trust your own certificates.
- Using a combination of
cert-managerand your own certificates. You can usecert-managerto automatically manage certificates for the internal services while providing your own certificates for the API, UI and secret vault, for example. Refer to the cert-manager guide to install cert-manager and then refer to the manual certificate management guide to configure your own certificates.
Remember that TLS certificates expire and need to be renewed. Depending on your chosen option, you will need to manually renew your certificates before they expire to ensure that your instance continues to function properly. Check the relevant guide for more information on how to renew your certificates.
What to do next
- Follow the guide for your chosen option to configure your certificates and proceed with the installation of OpsChain.