Step runner

This guide covers how the OpsChain server executes actions within the OpsChain step runner.

After reading this guide you should understand:

• how to create and use a custom step runner image
• how the API server and step runner exchange critical information

OpsChain runner images

Each step in an OpsChain change is executed inside a container that is based on an OpsChain runner image.

Each container only runs a single step before it is discarded. This ensures that:

• steps running in parallel do not impact each other
• modifications made by previously completed steps do not affect future steps
• the step execution environment contains only the current project's configuration and files

The image used by the step container is built as part of every step's execution and relies on build caching functionality to keep this performant.

The OpsChain runner base image is an AlmaLinux-based image that provides the standard RHEL-packaged base development tooling, a Ruby installation and the required Ruby Gems.

The runner image is called limepoint/opschain-runner and is configured by default for use in OpsChain.

Custom step runner Dockerfiles

If your resources or actions rely on external software, the image used by your project for its step runner containers can be modified to add extra packages or executables. The image may also be modified to optimise the performance of build steps by performing tasks as part of the step image build rather than as part of the step execution.

Creating a custom step runner Dockerfile

If your Git repository contains a Dockerfile in .opschain/Dockerfile, this will be used to build the image for your change's step runner containers. It must be based on the default step runner image Dockerfile to ensure compatibility with OpsChain. Download the sample Dockerfile, or get it by running:

curl -L https://docs.opschain.io/files/samples/2026-08-14/Dockerfile -o Dockerfile

It can also be retrieved from the OpsChain API server. (The benefit of retrieving it from the OpsChain server is that it is definitely the default Dockerfile for your version of OpsChain.) For example:

curl https://<opschain-host>/downloads/Dockerfile -o Dockerfile

This will download the file in the current directory. Make sure you move it to the .opschain directory in your Git repository.

Using the editor of your choice, make any desired modifications to the Dockerfile. See the customising the dockerfile and supported customisations sections below for more information.

Finally, add and commit the Dockerfile to your Git repository

git add .opschain/Dockerfile
git commit -m "Adding a custom Dockerfile."

NOTES

1. commits prior to this point won't use the custom Dockerfile because it is not present in the repository.
2. if you no longer wish to use the custom Dockerfile, .opschain/Dockerfile can be removed from the project repository.
3. the .opschain directory refers to the OPSCHAIN_REPO_FOLDER setting. If you have used a different value, use that instead.

Customising the Dockerfile

This Dockerfile can be modified and committed like any other file in the project Git repository.

The build context used when building the step runner image has access to the following files:

• repo.tar - The complete project Git repository, excluding the .git directory by default (see include_git_history to include it). This file will change (and invalidate the build context) when a different commit is used for a change or when there are changes to the project's Git repository

  tip

  See image performance - reducing build context size below for a way to exclude files from repo.tar that your steps don't need, to improve build performance.

• opschain-trust-store.tar - The certificate authorities you have uploaded, taken from the opschain-trust-store config map. The default Dockerfile extracts these into /etc/pki/ca-trust/source/anchors so that your build and your steps trust them. This file will change when a certificate authority is added or removed

• step_context.json.zip - A zip containing a step_context.json file with the environment variable properties of each owner contributing to the step, along with the parents and parent_order context values, for use by opschain-exec. This file will change if any of those environment variable properties change

• mintpress.license - The MintPress licence, taken from the mintpress-licence Kubernetes secret. It is supplied to the build as a BuildKit secret with the id mintpress_license, and the default Dockerfile mounts it at /opt/opschain/.environmint/mintpress.license. It is empty if no licence has been installed

step_context.json.zip is supplied to the build as a BuildKit secret with the id env_context_zip, rather than being copied into the image. Mount it on each RUN step that uses opschain-exec:

RUN --mount=type=secret,required=true,id=env_context_zip,uid=10001,gid=10001,target=/opt/opschain/.opschain/step_context.json.zip \
    opschain-exec <command>

OpsChain reads step_context.json from within the zip, so there is no need to extract it yourself. Without the mount, opschain-exec still runs the command, but none of the OpsChain environment variables will be set.

The build arguments supplied to BuildKit when building the image include:

Argument	Description
GIT_REV	The Git revision supplied to OpsChain when creating a change.
GIT_SHA	The Git SHA this revision resolved to at the time of creating the change.
OPSCHAIN_BASE_RUNNER	The system default base runner image (including image tag). (i.e. limepoint/opschain-runner:<OPSCHAIN_VERSION>).
OPSCHAIN_VERSION	The current OpsChain Docker image version.

The Dockerfile reference and the best practices for writing Dockerfiles guide provide more information about writing Dockerfiles.

Supported customisations

Modifying the Dockerfile allows a lot of flexibility.

For maximum compatibility with OpsChain we suggest only using the Dockerfile RUN, COPY, ENV, and ADD commands.

More advanced modifications (like modifying the ENTRYPOINT) are not supported and may break OpsChain.

Custom Dockerfiles must be based on an OpsChain base runner image (i.e. limepoint/opschain-runner) and we suggest using FROM ${OPSCHAIN_BASE_RUNNER} (as per the default Dockerfile) to achieve this.

Secure secrets

OpsChain allows users to leverage Kubernetes secrets to load sensitive information into the execution of steps and changes as environment variables.

When a property and a secret both define the same environment variable, the secret value will be used.

Secure build secrets

By default, OpsChain will supply the key value pairs configured in the opschain-build-env Kubernetes secret into the step runner image build. These are made available as environment variables to commands run via opschain-exec in your Dockerfile.

For example, if your custom step runner requires a utility from an AWS S3 drive, you can add your AWS credentials as key value pairs to the opschain-build-env secret (the default secret created for use by OpsChain image builds):

AWS_ACCESS_KEY_ID: QUtSQVFJQVpRUTdTRE9BSTM3NkYK
AWS_SECRET_ACCESS_KEY: djNLWll5RWtrbTd2NzBrOUFzRG04ZEFUQ1pZT0xMYWVsNXFwSWZFQwo=

These environment variables will then be available to the aws CLI (when run via opschain-exec), so it can authenticate to copy the utility, for example:

RUN --mount=type=secret,required=true,id=env_context_zip,uid=10001,gid=10001,target=/opt/opschain/.opschain/step_context.json.zip \
    opschain-exec aws s3 cp s3://source-bucket-name/customer-utility /opt/opschain/customer-utility

note

The --mount line is required. It gives opschain-exec access to the step context for this RUN step - without it the command runs with none of the OpsChain environment variables set.

More granular control over the secrets that are supplied to the image build is available by configuring the env:build_secrets configuration in the project or environment properties. See project and environment configuration for more information.

Secure runner secrets

By default, OpsChain will supply the key value pairs configured in the opschain-runner-env Kubernetes secret into the step runner container. These will be exported as environment variables when starting the step runner container.

For example, if your step copies a file into an AWS S3 drive, you can add your AWS credentials as key value pairs to the opschain-runner-env secret:

AWS_ACCESS_KEY_ID: QUtSQVFJQVpRUTdTRE9BSTM3NkYK
AWS_SECRET_ACCESS_KEY: djNLWll5RWtrbTd2NzBrOUFzRG04ZEFUQ1pZT0xMYWVsNXFwSWZFQwo=

note

Per Kubernetes requirements, the values in the secret must be base64 encoded.

These environment variables will then be available to the aws CLI when run as part of the OpsChain action, for example:

action :copy_utility do
  sh 'aws s3 cp build.war s3://destination-bucket-name/build.war'
end

More granular control over the secrets that are supplied to the step runner container is available by configuring the env:runner_secrets configuration in the project or environmentproperties. See project and environment configuration for more information.

Project & environment secret configuration

OpsChain allows you to configure specific secrets to supply to changes in an environment by configuring the env:build_secrets and env:runner_secrets configuration options in your project or environment properties.

For example, adding the following to the project and environment properties will cause OpsChain to provide the key value pairs in the project-build-secrets-1, project-build-secrets-2 and environment-build-secrets secrets as environment variables to opschain-exec during the image build:

Project properties:

{
  "opschain": {
    "env:build_secrets": ["project-build-secrets-1", "project-build-secrets-2"]
  }
}

Environment properties:

{
  "opschain": {
    "env:build_secrets": ["environment-build-secrets"]
  }
}

NOTES

1. Project and environment secrets are loaded in the order they are specified in your configuration - project secrets then environment secrets. If an environment variable exists in multiple Kubernetes secrets, the value from the most recently loaded secret will be supplied
2. If you have configured env:build_secrets in your project or environment configuration, the environment variables in the opschain-build-env secret will not be supplied to your image build. To include them, simply add opschain-build-env to the project or environment env:build_secrets configuration
3. If you have configured env:runner_secrets in your project or environment configuration, the environment variables in the opschain-runner-env secret will not be supplied to your step runner. To include them, simply add opschain-runner-env to the project or environment env:runner_secrets configuration

caution

The env:build_secrets and env:runner_secrets configuration options cannot be set in repository properties. If either option is configured in your project's repository properties, it will be ignored.

See the secrets tutorial for more information.

Image performance - base images

OpsChain runs the image build for every step within a change.

This is normally performant due to the image build cache - however it is possible to prebuild a custom base image if desired. This may make the image build faster when run for each step.

A custom base image can be created as follows:

1. Create a Dockerfile for the base image that uses FROM limepoint/opschain-runner.

   FROM limepoint/opschain-runner
   
   # run your custom build commands like any Dockerfile
   # Note: the OpsChain build context files will not be available here

2. Build and distribute the base image, assigning it a unique tag (the my-base-image used below is for example purposes only).

   docker build -t my-base-image .

3. Use the custom base image in the project custom Dockerfile.

   FROM my-base-image # supply the tag used above
   
   ... # the rest of the OpsChain custom Dockerfile

4. Run your change as normal. It will now use the my-base-image image as the base for the custom step image.

OpsChain relies on configuration done as part of the base runner image to work. By basing the custom base image on limepoint/opschain-runner the OpsChain configuration still applies and will work as desired.

Ensure that you rebuild your custom image after upgrading OpsChain.

Image performance - reducing build context size

As noted above, repo.tar contains your complete Git repository and is rebuilt for every step. Files that your steps don't actually need - for example documentation, test fixtures, large binary assets, or other unrelated content checked into the same repository - are still archived and hashed as part of every build, even if your Dockerfile never copies them into the final image.

Git provides a native way to exclude paths from git archive (and therefore from repo.tar): add an export-ignore entry to a .gitattributes file in your repository, for example:

docs/** export-ignore
test/fixtures/** export-ignore

Paths marked this way are excluded automatically, with no changes required to your custom step runner Dockerfile. For repositories with a lot of content unrelated to your OpsChain project files, this can noticeably improve build performance.

The .git directory itself (the full commit history) is a separate, often larger contributor to repo.tar's size. By default it is no longer included at all, since the default Dockerfile doesn't use it. If your custom Dockerfile relies on .git being present, enable the include_git_history setting.

API - step runner integration

When running the step runner, OpsChain includes:

1. the project's Git repository, reset to the requested revision, in the /opt/opschain directory
2. an /opt/opschain/.opschain/step_context.json file, containing the properties of each owner contributing to the step along with the current step's context values

Upon completion, the step will produce an /opt/opschain/.opschain/step_result.json file to be processed by the API server, detailing:

1. any changes to those properties the action has performed
2. the merged set of properties used by the action
3. any child steps to be run after this action (and their execution strategy)

Step context JSON

The step_context.json file supplied to the step includes the following sections:

JSON path	Description
context	The step context values for the current step - see the OpsChain context guide for more details
properties/<owner>	The properties converged for each owner contributing to the step, keyed by owner - project, environment, asset, template, template_version and change, as applicable to the change

The owners present in properties, and the order OpsChain merges them in, are listed in context/parent_order.

A sample step_context.json file is available to view here.

Rebuild any custom runner images after upgrading

This release changes the step_context.json format supplied to every step, so custom runner images built before upgrading no longer match what the current OpsChain server sends them. Rebuild any custom runner images after upgrading. Because OpsChain can reuse a previously built custom runner image, an instance that has already been upgraded can still pick up an image built before the upgrade and fail - rebuilding is required, not optional.

Step result JSON

The step_result.json file has the following structure:

{
  "step": {
    "properties": {
      "opschain": {}
    }
  },
  "properties_diffs": {
    "project": [
      {
        "op": "add",
        "path": "/new_element",
        "value": "test_value"
      }
    ],
    "environment": [],
    "change": []
  },
  "child_step_definitions": {
    "children": [
      {
        "action": "sample:hello_world_1:run"
      }
    ],
    "child_execution_strategy": "sequential"
  },
  "expected_step_tree": null
}

File content - step result

The step/properties contains the merged set of properties applied to the action. These are linked to the step to support future investigation / debugging.

The properties_diffs value is keyed by property owner - the same owners supplied to the step in step_context.json. Each value is a set of RFC6902 JSON Patch operations, describing the changes to apply to that owner's properties.

The child_step_definitions value contains the child steps (and execution strategy) the OpsChain workers will execute.

The expected_step_tree value contains the step tree OpsChain derived from the action's definition in your actions.rb - its step name, description, prerequisites and children. OpsChain uses this to reconcile the running step's details and to build the change's step tree.

Log messages for step phases

OpsChain includes log messages in your change logs to allow you to follow each step's progress as OpsChain builds its step runner and executes the step's action. These messages will log when the phase starts, initialises (if relevant), is completing (if relevant), and finishes. These log messages can be used to diagnose how much time the different phases of a change/step are taking.