Pre-installation requirements for Google Kubernetes Engine

For details about supported platforms for CloudBees CI on modern cloud platforms, such as supported Kubernetes, Helm, and NFS versions, refer to Supported platforms for CloudBees CI on modern cloud platforms.

Google Kubernetes Engine requirements

The following items are required to install CloudBees CI on modern cloud platforms on Google Kubernetes Engine (GKE):

On your local computer or a bastion host:
- A Kubernetes client with a currently supported version of Kubernetes, installed and configured (kubectl). Beta releases are not supported.
- gcloud (See Installing Google Cloud SDK for instructions).
A GKE cluster running a currently supported version of Kubernetes. Beta releases are not supported.
- With nodes that have at least 2 CPUs, 4 GiBs of memory (so nodes have 1 full CPU / 1 GiB available after running a master with default settings).
- The cluster must have network access to container images (public Docker Hub or a private Docker Registry).
A namespace in the cluster (provided by your admin) with permissions to create Role and RoleBinding objects.
Kubernetes cluster Default Storage Class defined and ready to use.
- Refer to the Kubernetes on Google Kubernetes Engine reference for more information.

Storage requirements

Dynamic provisioning is required to create persistent volumes. If you don’t enable dynamic provisioning, you will have to manually create a persistent volume.

Because Jenkins is highly dependent upon the filesystem, the underlying storage provider must provide sufficient input/output operations per second (IOPS), sufficient throughput, and low latency.

SSD disks are recommended for production usage.

Table 1. Storage recommendations for GKE
	Single AZ	Multiple AZ	High Availability (HA)
Zonal Persistent Disk
Regional Persistent Disk
Filestore

Install High Availability on GKE

To install High Availability in GKE:

Set up a storage class with ReadWriteMany access mode.

CloudBees recommends setting this up in Google Filestore. For more information, refer to the Google Cloud Platform (GCP) documentation on the Filestore CSI driver. Be sure to use an SSD-based service tier, as HDD latencies could result in poor performance.

Prepare the Helm values. The following example can be used as a starting point:

OperationsCenter:
  HostName: cloudbees-ci.example.com
  Protocol: https
  Platform: gke
Gateway:
  Enabled: true
  Name: '<gateway-name>'
  Namespace: '<gateway-namespace>'
Persistence:
  # default from `gcloud container clusters create … --addons=GcpFilestoreCsiDriver`:
  StorageClass: premium-rwx▼

Managed controllers, as described in HA and caches in CloudBees CI on modern cloud platforms, automatically set cache-related properties to use per-replica temporary folder locations.

It is possible to use generic ephemeral volumes for this purpose, and, in Google Kubernetes Engine, you can define SSD disks for the ephemeral storage following these steps:

Create a node pool with SSD disks:

gcloud container node-pools create with-ssd --cluster=<your-cluster-name> --ephemeral-storage-local-ssd count=1 --machine-type=<your-machine-type> --region=<your-region>(1)▼

1	Replace `<your-cluster-name>`, `<your-machine-type>`, and `<your-region>` with the appropriate values.

Customize your High Availability managed controller to use nodes using the SSD disks:

containers:
      - name: jenkins
        ...
        resources:
          requests:
            ephemeral-storage: "300Gi"
        ...
nodeSelector:
  cloud.google.com/gke-ephemeral-storage-local-ssd: true
...▼

Refer to the Ephemeral storage as an emptyDir volume documentation page for more information.

Gateway API requirements

CloudBees CI uses the Kubernetes Gateway API to manage traffic routing. When Gateway API is enabled, CloudBees CI creates HTTPRoute resources for the operations center and all managed controllers.

Any conformant Gateway API implementation is supported. For tested implementations, refer to Kubernetes Gateway API supported implementations.

For the complete list of Gateway API prerequisites, refer to Verify Kubernetes Gateway API prerequisites.

Gateway API manages HTTP/HTTPS traffic only. CloudBees recommends WebSocket transport for external inbound agent communication, which uses the standard HTTP/HTTPS port and requires no additional configuration. For setup instructions, refer to Use WebSockets to connect controllers to the operations center.

If you use a Gateway API implementation that CloudBees has not tested, additional configuration may be required for TLS termination, session persistence, or namespace routing. Refer to the CloudBees CI Knowledge Base for guidance. CloudBees does not provide support for untested implementations. You must install and configure your Gateway API controller and adjust your CloudBees CI chart values according to your situation.

For High Availability (HA) controllers, refer to session persistence implementation support to verify Gateway API session persistence compatibility with your implementation.

Set up a Gateway API controller

A supported Kubernetes Gateway API controller is required. For more information, refer to Kubernetes Gateway API supported implementations and Verify Kubernetes Gateway API prerequisites.

On Google Kubernetes Engine (GKE), you can use the built-in GKE Gateway controller (ensure Gateway API is enabled on the cluster) or deploy a third-party implementation such as Istio. Refer to the GKE Gateway documentation for setup instructions.

If you use a third-party implementation, the GKE-specific resources described below do not apply; follow the instructions for your chosen implementation instead.

GKE Gateway controller custom resources

The information in this section was verified using an isolated Google Cloud Platform (GCP) project with no other services connected to the cluster.

When using the built-in GKE Gateway controller, you must create vendor-specific custom resources from the networking.gke.io API group.

Health checks with `HealthCheckPolicy`

GKE Gateway controller does not infer health check parameters from Kubernetes readiness probes. The default health check sends GET / and expects HTTP 200, but the operations center and controllers return a redirect, so GKE marks backends as unhealthy and all requests fail.

You must create a HealthCheckPolicy resource for every Service behind the Gateway, including the operations center and each controller. Set the health check path to /health/, prefixed with the context path in path-based routing mode (for example, /cjoc/health/ for the operations center, /<controller-name>/health/ for a controller).

For controllers, add the HealthCheckPolicy to the YAML configuration section of the controller provisioning configuration so it is created at provision time and removed at deprovision. For the operations center, create the HealthCheckPolicy manually after Helm install, targeting the cjoc Service.

For the resource schema and examples, refer to the GKE health check configuration documentation.

Session affinity for HA controllers with `GCPBackendPolicy`

GKE Gateway controller silently ignores the standard sessionPersistence field on HTTPRoute. If you run HA controllers, you must create a GCPBackendPolicy resource with cookie-based session affinity (GENERATED_COOKIE) for each HA controller Service. This follows the same workaround pattern as the Istio DestinationRule described in Configure sticky sessions with Istio.

Add the GCPBackendPolicy alongside the HealthCheckPolicy in the YAML configuration section of the controller provisioning configuration.

Session affinity applies only to HA controllers. Standard (non-HA) controllers do not require it.

For the resource schema and examples, refer to the GKE session affinity configuration documentation.

Role-Based Access Control for GKE custom resources

The operations center ServiceAccount must have permissions to create, update, and delete healthcheckpolicies and gcpbackendpolicies resources in the networking.gke.io API group. Without these permissions, the operations center cannot create the resources via the additional YAML in the controller provisioning configuration. Create a Role and RoleBinding granting these permissions, following the same pattern as the Istio RBAC example.

Create your GKE cluster

To create a GKE cluster refer to the official Google documentation Create a GKE cluster.

More information on administering a Google Kubernetes cluster is available from the Kubernetes Engine How-to Guides.

More information on Kubernetes concepts is available from the Kubernetes site, including: