Installing on Kubernetes

CloudBees simplifies the installation of CloudBees Jenkins Distribution on Kubernetes by providing you with a Helm chart.

Prerequisites

At a minimum, the Kubernetes cluster requirements must be satisfied before CloudBees Jenkins Distribution can be installed.

CloudBees Jenkins Distribution is supported only on production releases of Kubernetes and Helm.

System and platform prerequisites

Refer to Supported platforms for CloudBees Jenkins Distribution for details about platforms supported for CloudBees Jenkins Distribution. This section provides details specific to installing on Kubernetes.

CloudBees Jenkins Distribution admin workstation

The CloudBees Jenkins Distribution admin workstation is the computer used to install, update, and maintain CloudBees Jenkins Distribution.

This workstation may be the workstation for the CloudBees Jenkins Distribution administrators or a Kubernetes administrator.

In organizations that have several individuals performing CloudBees Jenkins Distribution administrator duties, it may be beneficial to use a bastion host instead of setting up a workstation for each CloudBees Jenkins Distribution administrator.

The requirements for the admin workstation are:

  • The kubectl Kubernetes Control client application, version 1.11 or greater, installed and configured to work with your Kubernetes cluster.

  • The Helm client. Helm requirements are described later in the Helm section.

Supported Kubernetes

For CloudBees Jenkins Distribution, CloudBees supports Kubernetes 1.x, starting with 1.11, as long as it is actively supported by the Kubernetes distribution provider and is one of the following implementations:

  • Amazon EKS

    The version used must be Generally Available. CloudBees does not support or recommend "proof of concept" offering of Kubernetes platforms or Beta or Public preview versions.

    CloudBees Jenkins Distribution might work on other Kubernetes implementations such as Azure Kubernetes Service (AKS) or Google Kubernetes Engine (GKE) if it is installed as a Kubernetes Helm Chart, but CloudBees does not fully support these platforms. Red Hat OpenShift is not supported for the moment.

CloudBees only supports CloudBees Jenkins Distribution on Kubernetes when it is installed using the official CloudBees Jenkins Distribution Helm charts.

To create an Amazon Kubernetes cluster (EKS), refer to the official Amazon EKS documentation.

More information on Kubernetes concepts is available from the Kubernetes site, including:

Kubernetes cluster

CloudBees Jenkins Distribution must run on a Kubernetes cluster, and the cluster must meet these requirements:

  • A production release of Kubernetes 1.11 or greater, as long as it is actively supported by the Kubernetes distribution provider and generally available, with:

    • Nodes that have at least 2 CPUs and 4 GBs of memory.

    • Kubernetes cluster nodes in each availability zone.

  • Network access to container images (public Docker Hub or a private Docker Registry).

  • A namespace in the cluster (provided by your Kubernetes admin) with permissions to create Role and RoleBinding objects.

  • Access to the DNS record that points to your installation.

  • SSL certificates (needed when you deploy CloudBees Jenkins Distribution):

    • For production environments, install your own NGINX Ingress controller and obtain TLS certificates. Refer to NGINX Ingress controller section for further details.

    • If you are not planning to use this installation as a production environment, you can install the default NGINX Ingress Controller without enabling TLS. However, this means that only HTTP will be supported and not HTTPS.

  • A Default Storage Class defined and ready to use

    • To identify your default storage class, use the kubectl get sc -o yaml | grep storageclass.beta.kubernetes.io/is-default-class command.

Supported Helm

CloudBees supports only production releases of Helm: RC, beta, patch or experimental releases of Helm are not supported.

  • Starting with CloudBees CI on modern cloud platforms version 2.204.3.4, Helm version 3.0.2 or later

Helm V2

  • Helm client, production release version 2.x, starting with version 2.12, installed locally.

  • If you are using Helm to manage your application deployment: Tiller server production release version 2.x, starting with version 2.12, installed on your Kubernetes cluster.

We recommend that you migrate to Helm V3 as soon as possible. Support of Helm V2 will be deprecated in July 2020.

NGINX Ingress controller

CloudBees Jenkins Distribution requires an NGINX Ingress controller. CloudBees Jenkins Distribution can either use an existing NGINX Ingress Controller or you can install an NGINX Ingress Controller as part of the Helm chart installation process.

Although the NGINX Ingress Controller installed with the Helm chart meets CloudBees Jenkins Distribution requirements, it is not ideal for a production environment; if you are operating a production environment that requires TLS, you should create the NGINX Ingress Controller before installing the Helm chart.

+ Instructions for installing the NGINX Ingress Controller are given in Installing CloudBees CI on Amazon Elastic Kubernetes Service (Amazon EKS), and the NGINX Ingress Controller Installation Guide gives instructions for customizing the installation.

Helm

Helm is a package manager for Kubernetes that manages an application’s dependencies, installation, and configuration.

Helm manages the process of customizing an application to a specific environment and dramatically simplifies installation, upgrades, rollbacks, and migrations.

If you already have Helm client and Tiller server-side component installed, you can skip to the CloudBees Helm chart repository section.

Helm terminology

Helm has two components: * A Helm client. * A server component, called Tiller.

The Kubernetes administrator uses the Helm client to issue commands to the Tiller server.

A Helm installation is called a release. Helm can automatically create a release name. CloudBees recommends using a custom name such as cloudbees-jenkins-distribution during your initial install.

Every Helm command that is applied to the release is stored as a revision.

CloudBees recommends using Helm with the Tiller server-side component.

Installing the Helm client

The Helm project readme provides detailed instructions on installing the Helm client on various operating systems in the installation section.

Installing Tiller

Tiller is the Kubernetes cluster’s server-side component of Helm.

Use the following steps to install Tiller.

  1. Create a service account for Tiller:

    shell% kubectl create serviceaccount --namespace kube-system tiller
  2. Give the service account administrative capabilities:

    shell% kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller
  3. Install Tiller on the Kubernetes cluster:

    shell% helm init --service-account tiller

Installing CloudBees Jenkins Distribution using Helm

CloudBees hosts the Helm chart on CloudBees' public Helm chart repository. To access it, you need to add the repository to your Helm environment.

Adding the CloudBees Helm chart repository

Add the CloudBees Helm chart to your Helm environment with the helm repo add command:

$ helm repo add cloudbees https://charts.cloudbees.com/public/cloudbees (1)
$ helm repo update (2)
1 The helm repo add adds a new Helm chart repository to your Helm installation.
2 The helm repo update updates your local Helm chart repository cache. Your local Helm chart repository cache is used by helm commands like helm search to improve performance.

Creating and using a namespace

CloudBees recommends using Kubernetes namespaces to install CloudBees Jenkins Distribution.

Combining Kubernetes RBAC security with a namespace allows a Kubernetes administrator to restrict who has access to a namespace and its data.

Your Helm release name must be unique for the Kubernetes cluster, not just your namespace.

To create a namespace:

+

shell% kubectl create namespace cloudbees-jenkins-distribution
shell% kubectl config set-context $(kubectl config current-context) --namespace=cloudbees-jenkins-distribution

Performing a default installation

The default installation accepts all the default values CloudBees has defined and requires the NGINX Ingress controller:

+

shell% helm install --name cloudbees-jenkins-distribution \ (1) (2)
               --namespace cloudbees-jenkins-distribution \ (3)
               cloudbees/cloudbees-jenkins-distribution
1 If you copy this command into your terminal don’t forget to remove the annotations at the end of each line.
2 The name isn’t required, but without it Helm will automatically generate a release name with random strings. For this reason, CloudBees recommends always setting a release name.
3 Adding the --namespace argument instructs Helm to install everything in the cloudbees-jenkins-distribution namespace otherwise, it uses the currently selected namespace. Remember that the namespace must already exist.

The command deploys cloudbees-jenkins-distribution on the Kubernetes cluster in the default configuration.

The Configuration options section lists the parameters that can be configured during installation.

Installing CloudBees Jenkins Distribution with an NGINX Ingress controller

The chart is designed so it can install an NGINX Ingress controller. The nginx-ingress.enabled field controls if the Ingress controller has to be installed installation and setup. The example below illustrates how to install the chart with the release name cloudbees-jenkins-distribution and hostname cloudbees-jenkins-distribution.example.com.

$ helm install --name cloudbees-jenkins-distribution\ (1)
               --set nginx-ingress.enabled=true \ (2)
               --set master.serviceType='ClusterIP' \ (3)
               --namespace cloudbees-jenkins-distribution \ (4)
               cloudbees/cloudbees-jenkins-distribution
1 Just like the previous example, the release name is set to cloudbees-jenkins-distribution.
2 Setting nginx-ingress.enabled to true causes the CloudBees Jenkins Distribution Helm chart to also install a NGINX Ingress controller using the NGINX Ingress Helm chart.
3 Since we want to use an ingress and the NGINX Ingress controller we need to use ClusterIP master.serviceType.
4 Adding the --namespace argument instructs Helm to install everything in the cloudbees-jenkins-distribution namespace. Otherwise, it uses the currently selected namespace. Remember the namespace must already exist.

Using Custom Property Value files

Helm provides several ways to set value fields. The CloudBees Jenkins Distribution Helm chart provides CloudBees’s default values for CloudBees Jenkins Distribution. Helm stores these values in the chart’s values.yaml file. In the preceding sections, we’ve demonstrated overriding these fields on as helm install arguments using the --set parameter.

The final option is to create a custom YAML file containing value field assignments to override the default values set in the CloudBees Jenkins Distribution values.yaml file for your environment. CloudBees recommends creating a custom values file for your installation to keep all of your CloudBees Jenkins Distribution environment settings in your version control system to track of changes to the environment.

CloudBees recommends using a custom values file to upgrade and install CloudBees Jenkins Distribution using Helm.
helm install cloudbees/cloudbees-jenkins-distribution \
             --name cloudbees-jenkins-distribution \
             -f override-values.yaml \ (1)
             --namespace=cloudbees-jenkins-distribution
1 To use the custom value file with Helm, you use the -f argument with the custom value filename.

Upgrading CloudBees Jenkins Distribution using Helm

One of the key features of using Helm is the ease of upgrading and updating your CloudBees Jenkins Distribution release. The helm upgrade command allows you to upgrade your CloudBees Jenkins Distribution release. This can be new setting or a new version of CloudBees Jenkins Distribution.

Helm Issue 2948 and NGINX Ingress controller support

If you are using Installing CloudBees Jenkins Distribution with an NGINX Ingress controller option, you MUST set the nginx-ingress.enabled=true or the NGINX Controller will be deleted. For this reason, CloudBees strongly recommends you use a custom values file. For more details around Helm Issue 2948, see Helm Issue 2948 on GitHub.

This can be a new version of the CloudBees Jenkins Distribution Helm chart or an updated value you wish to apply to you release.

helm upgrade -f upgrade-values.yaml --dry-run --debug (1) (2)
1 Run the helm upgrade command with the -f argument to use the upgrade-value.yaml file Replace the examples-value.yaml filename with the name of your values file.
2 Adding the --dry-run --debug run dry run of the installation without making changes.

Configuration options

The following tables lists the configurable parameters of the CloudBees Jenkins Distribution chart and their default values. Many of them are based on the ones provided by the Jenkins OSS Helm chart and CloudBees provides recommended values for these options based on our experience.

Master configuration values

Table 1. Operations Center configurations values
Parameter Description Default

nameOverride

Override the resource name prefix

cloudbees-jenkins-distribution

fullnameOverride

Override the full resource names

cloudbees-jenkins-distribution-{release-name} (or cloudbees-jenkins-distribution if release-name is cloudbees-jenkins-distribution or cjd)

master.numExecutors

Set number of executors

0

master.adminUser

Administrator user of the CloudBees Jenkins Distribution instance

admin

master.adminPassword

Initial admin user password. If the password isn’t set one will be automatically generated.

Autogenerated random value

master.jenkinsAdminEmail

Email address for the administrator of the CloudBees Jenkins Distribution instance

Not set

master.hostAliases

Aliases for IPs to add in Pod /etc/hosts

[]

master.serviceType

k8s service type

LoadBalancer

master.servicePort

Service port for CloudBees Jenkins Distribution

8080

master.targetPort

Port where CloudBees Jenkins Distribution is running

8080

master.nodePort

Port where CloudBees Jenkins Distribution is accessible

Not set

master.agentListenerPort

Listening port for agents

50000

master.agentHostPort

Host port to listen for agents

Not set

master.disabledAgentProtocols

Disabled agent protocols

JNLP-connect JNLP2-connect

master.csrf.defaultCrumbIssuer.enabled

Enable the default CSRF Crumb issuer

true

master.csrf.defaultCrumbIssuer.proxyCompatability

Some HTTP proxies filter out information that the default crumb issuer uses to calculate the nonce value. Seting a true value makes the nonce value easier to forge.

true

master.cli

Enable CLI over remoting

false

master.loadBalancerSourceRanges

Allowed inbound IP addresses. Only in case the serviceType is LoadBalancer. By default all IPs are allowed.

0.0.0.0/0

master.jmxPort

Open port for JMX stats

Not set

master.extraPorts

Open extra ports in the container running in the Pod if needed

Not set

master.overwriteConfig

Override config.xml, jenkins.CLI.xml, and jenkins.model.JenkinsLocationConfiguration.xml (only recommended for advanced admin users)

false

master.jenkinsUrlProtocol

Set protocol to access CloudBees Jenkins Distribution as part of JENKINS_URL value

Set to https if Master.ingress.tls, http otherwise

master.initScripts

List of scripts to execute during the initialization

Not set

master.credentialsXmlSecret

Kubernetes secret that contains a credentials.xml file. Use only if credentials plugin is installed and only in upgrades.

Not set

master.secretsFilesSecret

secrets files to add to the instance

Not set

master.scriptApproval

List of groovy functions to approve in case script-security plugin is installed. Use only to add new scripts in upgrades.

Not set

master.affinity

Set Affinity for preferred node selection. See Affinity in Kubernetes for further information.

{}

master.jenkinsUriPrefix

Root Uri where CloudBees Jenkins Distribution will be served on. It is appended to JENKINS_URL.

Not set

master.priorityClassName

The name of a priorityClass to apply to the master pod. See PriorityClass in Kubernetes for further information.

Not set

Agent values

Table 2. Agent Configurations Values
Parameter Description Default

agent.customJenkinsLabels

Append Jenkins labels to the agent

{}

agent.enabled

Enable Kubernetes plugin jnlp-agent podTemplate

true

agent.imagePullSecret

Agent image pull secret

Not set

agent.privileged

Agent privileged container

false

agent.volumes

Additional volumes

Not set

agent.envVars

Environment variables for the Agent Pod

Not set

agent.command

Executed command when side container starts

Not set

agent.args

Arguments passed to executed command

Not set

agent.sideContainerName

Side container name in agent

jnlp

agent.TTYEnabled

Allocate pseudo tty to the side container

false

agent.containerCap

Maximum number of agent

10

agent.podName

Agent Pod base name

Not set

agent.idleMinutes

Allows the Pod to remain active for reuse

0

Persistence

Table 3. Persistence Configuration Values
Parameter Description Default

persistence.existingClaim

Instead of creating a new Persistent Volumes Claim, the CloudBees Jenkins Distribution will use this existing Persistent Volumes Claim

Not Set

persistence.size

The size of the PVC

8Gi

persistence.volumes

If defined, other volumes are mounted on the PVC

Not Set

persistence.mounts

If defined, other mount paths are mounted

Not Set

NGINX Ingress

The NGINX Ingress properties provided the option to install an NGINX Ingress controller. The default option of false, does not install the NGINX Ingress controller and will require an existing NGINX Ingress controller to be installed.

Table 4. NGINX Ingress configuration values
Parameter Description Default

nginx-ingress.enabled

Setting this property to true installs NGINX Ingress controller in addition to CloudBees Jenkins Distribution.

false

master.ingress.apiVersion

Ingress API version

extensions/v1beta1

master.ingress.hostName

Ingress host name

Not set

master.ingress.annotations

Ingress annotations

{}

master.ingress.labels

Ingress labels

{}

master.ingress.path

Ingress path

Not set

master.ingress.tls

Ingress TLS configuration

[]

master.route.enabled

Enables openshift route

false

master.route.annotations

Route annotations

{}

master.route.labels

Route labels

{}

master.route.path

Route path

Not set