Use Buildkit with CloudBees CI

Buildkit is Docker builder that provides new functionality and improves the performance of your builds. Docker buildx provides a Kubernetes driver to build docker images in Kubernetes pods using Buildkit. In comparison to Kaniko, Buildkit does not require being run as root; it’s considered to be a more secure way to build Docker images in Kubernetes.

If Pod Security Admission (PSA) restricted is enabled, Buildkit will not function.

Pipeline example

This example illustrates running a Pipeline with Buildkit installed.

Requirements

To run this example, you need the following:

A Kubernetes cluster with an installation of CloudBees CI
Ability to run kubectl against your cluster
CloudBees CI account with permission to create the new pipeline

Additional configuration is needed to run multi-architecture builds. That is not covered in this article. Refer to Building multi-platform images.

Steps

These are the high-level steps for this example:

Install Buildkit in the cluster.
Create the Pipeline.
Run the Pipeline.

Install Buildkit in the cluster

To install Buildkit in the cluster, follow the instructions in the official documentation.

Kubernetes setup buildx

$ kubectl create namespace buildkit
$ docker buildx create \
  --bootstrap \
  --name=buildkit \
  --driver=kubernetes \
  --driver-opt=namespace=buildkit,rootless=true▼

The --name=buildkit flag sets the builder name. The kubernetes driver uses this name to create a Deployment named buildkit0 in the buildkit namespace. The --bootstrap flag provisions this Deployment immediately, as a one-time administrator action.

The builder name (--name) and driver options (--driver-opt) must match the values used in the pipeline (refer to Run the pipeline). The pipeline reuses this Deployment by name; if the names do not match, the pipeline attempts to create a new Deployment at runtime and fails with an RBAC error, because the agent service account lacks permission to create deployments.

Create the Pipeline

Grant the jenkins-agents service account permissions to use Buildkit, by creating this Role and RoleBinding in the buildkit namespace:

Role to grant access to the buildkit namespace

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: can-build
rules:
  - apiGroups:
      - apps
    resources:
      - deployments
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - pods
    verbs:
      - list
  - apiGroups:
      - ""
    resources:
      - pods/exec
    verbs:
      - create▼

RoleBinding to grant access to the buildkit namespace

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: agents-can-build
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: can-build
subjects:
  - kind: ServiceAccount
    name: jenkins-agents
    namespace: jenkins-agents-namespace (1)▼

1	The `<jenkins-agents-namespace>` is the namespace where agents run. The namespace value depends on your installation. Replace it with the right value from your CloudBees CI installation.

This Role only grants the permissions the agent requires. An administrator pre-provisions the BuildKit Deployment (refer to Install Buildkit in the cluster) and the pipeline reuses it by the same builder name. The agent only needs to get the existing Deployment and exec into its pods; it does not need permission to create deployments.

Run the pipeline

Run docker buildx build … in a pipeline as shown in the following example.

Pipeline example using Docker buildx

podTemplate(yaml: <your-pod-definition>) { (1)
    node(POD_LABEL) {
        git ... (2)
        sh '''
        docker buildx create --name buildkit --driver=kubernetes --driver-opt=namespace=buildkit,rootless=true --use
        docker buildx build --progress plain -t local-test:1 .
        '''
    }
}▼

1	`<your-pod-definition>` should use a Docker image which has the Docker and buildx extensions.
2	Clone a repository with a Dockerfile in the root folder.

The docker buildx create command in the pipeline uses the same --name and --driver-opt values as the one in Install Buildkit in the cluster, so both reference the same Deployment (buildkit0) that the administrator already provisioned. It differs only in --use, which makes this builder active for the docker buildx build that follows.

Both commands are required because docker buildx create writes to the local, per-machine buildx state. The administrator runs it once on their workstation, but each agent pod is ephemeral and must re-declare the builder locally. Because the names match, the pipeline reuses the existing Deployment rather than attempting to create a new one.