Install CloudBees CI on modern cloud platforms in FIPS mode

This guide explains how to install CloudBees CI on modern cloud platforms in FIPS mode into an Elastic Kubernetes Service (EKS) cluster on AWS GovCloud.

Prerequisites

AWS GovCloud account
FIPS compliant Istio (refer to Tetrate for more information)

Pre-installation

AWS does not provide a FIPS compliant AMI (Amazon Machine Image) for use as an EKS node. Follow the steps below to create a FIPS compliant AMI.

Get an AWS GovCloud account.

Create a FIPS AMI.

Download, install, and setup the AWS CLI.
Clone the amazon-eks-ami directory.

In that directory, run the following command:

make k8s=1.28 enable_fips=true source_ami_owners=045324592363 aws_region=us-gov-east-1 (1)▼

The value for k8s should be the latest Kubernetes version supported by the version of CloudBees CI that you are installing. Refer to Supported platforms for CloudBees CI on modern cloud platforms for more information.

If you only have access to an AWS account in GovCloud and not a AWS regular account, you may encounter an error obtaining EKS binaries from GovCloud. To resolve the issue, follow steps described here.

Use the resulting AMI for node_groups in your EKS cluster.

Install AWS Load Balancer.
Create a certificate in the AWS Certificate Manager.
Set up an External-DNS on AWS.
Install the Kubernetes Gateway API.

For more information about the Gateway API, refer to https://kubernetes.io/docs/concepts/services-networking/gateway/.

Configure your environment

Complete the following steps to configure your environment for the CloudBees CI on modern cloud platforms installation.

CloudBees CI on modern cloud platforms uses OCI container images. You can optionally Verifying the CloudBees CI on modern cloud platforms Docker images to ensure their origin and authenticity.

Add the CloudBees Helm chart repository as follows:

helm repo add cloudbees https://public-charts.artifacts.cloudbees.com/repository/public/
helm repo update▼

Before initiating an installation, configure the namespaces to enforce Istio mTLS and use the correct Istio control pane.

Run the following command to enforce the Istio mTLS.

kubectl apply -n istio-system -f - << EOF
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
spec:
  mtls:
    #mode: PERMISSIVE
    mode: STRICT
EOF▼

Run the following command to label the cloudbees-core namespace to use a specific Istio revision.

kubectl label namespace cloudbees-core istio-injection
kubectl label namespace cloudbees-core istio.io/rev=${ISTIO_REVISION_TAG} --overwrite▼

Create a file named values.yaml with the following content:

fips140: true
OperationsCenter:
  Platform: eks
  Gateway:
    Name: cbci-gateway
    Namespace: istio-ingress (1)
  CasC:
    Retriever:
      Enabled: false
Hibernation:
  Enabled: false
Agents:
  SeparateNamespace:
    Enabled: true
    Create: false (2)▼

1	This value should reference the Gateway created during Pre-installation.
2	The `Create: false` means the namespace has been created previously.

Checking Istio and Kubernetes gateway

When gathering cluster information for support, the Istio and Gateway API version is crucial when in FIPS mode.

To gather required information on the gateway, run the following commands:

# Check namespaces
$ kubectl get namespaces
NAME                               STATUS   AGE
...
istio-ingress                      Active   XXX
istio-system                       Active   XXX
...

# Check the gateway
$ kubectl get gateways --namespace istio-ingress
NAMESPACE       NAME           CLASS ...
istio-ingress   cbci-gateway   istio

# Get all the info from the gateway
$ kubectl get gtw cbci-gateway --namespace istio-ingress -o json
...

# Or get only basic info
$  kubectl get gtw cbci-gateway --namespace istio-ingress -o go-template='{{printf " apiVersion: %s\n" .apiVersion}} {{printf "Name: %s\n" .metadata.name}} {{printf "Kind: %s\n" .kind}} {{printf "Gateway ClassName: %s\n" .spec.gatewayClassName}}'
 apiVersion: gateway.networking.k8s.io/v1
 Name: cbci-gateway
 Kind: Gateway
 Gateway ClassName: istio▼

To gather data on the Istio version and images, run the following commands:

# Get existing deployment
$ kubectl get deployment --namespace istio-system
NAME            READY   UP-TO-DATE   AVAILABLE   AGE
istiod-1-20-3   1/1     1            1           XXX

# Get revision
$ kubectl get deployment.apps/istiod-1-20-3 --namespace istio-system -o "jsonpath={.metadata.labels}" | jq
{
  "app": "istiod",
  "app.kubernetes.io/managed-by": "Helm",
  "install.operator.istio.io/owning-resource": "unknown",
  "istio": "pilot",
  "istio.io/rev": "1-20-3",
  "operator.istio.io/component": "Pilot",
  "release": "istiod"
}

# Get image
$ kubectl get deployment.apps/istiod-1-20-3 --namespace istio-system -o "jsonpath={.spec.template.spec.containers[0].image}"
docker.io/istio/pilot:1.20.3

# Or simply get all available data
$ kubectl get deployment.apps/istiod-1-20-3 --namespace istio-system -o json
...

To gather data on the Istio ingress, run the following commands:

# Get existing deployment
$ kubectl get deployment --namespace istio-ingress
NAME                 READY   UP-TO-DATE   AVAILABLE   AGE
cbci-gateway-istio   1/1     1            1           XXX

# Get revision
$ kubectl get deployment.apps/cbci-gateway-istio --namespace istio-ingress -o "jsonpath={.spec.template.metadata.annotations}" | jq
{
  "cert-manager.io/issuer": "zerossl",
  "istio.io/rev": "1-20-3",
...
}

# Get image
$ kubectl get deployment.apps/cbci-gateway-istio --namespace istio-ingress -o "jsonpath={.spec.template.spec.containers[0].image}"
docker.io/istio/proxyv2:1.20.3

# Or simply get all the ingress data
$ kubectl get deployment.apps/cbci-gateway-istio --namespace istio-ingress -o json
...

Using AWS Elastic Container Registry (ECR) with CloudBees CI

You can install CloudBees CI from your ECR registry, as long as your cluster has access to ECR. To install CloudBees CI from your ECR registry, update the Helm chart so that it pulls CloudBees CI container images from ECR instead of the public Docker registry.

To setup AWS ECR to store the container images for use during your CloudBees CI on modern cloud platforms installation:

Create a registry in ECR from the AWS Console or the AWS SDK. The registry hostname should be formatted like this:
```
# {id-number}.dkr.ecr.{region}.amazonaws.com
123456789012.dkr.ecr.us-gov-east-1.amazonaws.com▼
```
Run the following commands to obtain a list of the CloudBees CI container images included in the Helm chart:
```
helm repo update
helm show values cloudbees/cloudbees-core --jsonpath="{..Image.dockerImage}"▼
```
Pull the CloudBees CI container images from Docker Hub.

To continue with the previous example, use the following commands to pull the CloudBees CI container images for version 2.176.1.4:
```
$ docker pull cloudbees/cloudbees-cloud-core-oc-fips:2.176.1.4
$ docker pull cloudbees/cloudbees-core-mm-fips:2.176.1.4
$ docker pull cloudbees/cloudbees-core-agent-fips:2.176.1.4▼
```

Run the following commands to tag the container images. When tagging the container images, replace ecr-repo with your registry hostname. For example:

$ docker tag cloudbees/cloudbees-cloud-core-oc-fips:2.176.1.4 {ecr-repo}/cloudbees/cloudbees-cloud-core-oc-fips:2.176.1.4
$ docker tag cloudbees/cloudbees-core-mm-fips:2.176.1.4 {ecr-repo}/cloudbees/cloudbees-core-mm-fips:2.176.1.4
$ docker tag cloudbees/cloudbees-core-agent-fips:2.176.1.4 {ecr-repo}/cloudbees/cloudbees-core-agent-fips:2.176.1.4▼

Run the following commands to push the container images to ECR. Replace the values below with the specific names of your container images:

$ docker push {ecr-repo}/cloudbees/cloudbees-cloud-core-oc-fips:2.176.1.4
$ docker push {ecr-repo}/cloudbees/cloudbees-core-mm-fips:2.176.1.4
$ docker push {ecr-repo}/cloudbees/cloudbees-core-agent-fips:2.176.1.4▼

Add the following to the values.yaml file, replacing {ecr-repo} with the actual host name:
```
Common:
  image:
    registry: {ecr-repo}/cloudbees▼
```
Now install CloudBees CI on modern cloud platforms in FIPS mode.

Install CloudBees CI on modern cloud platforms in FIPS mode

Type the following command to complete the installation:

helm install cloudbees-core \
               cloudbees/cloudbees-core \
               --namespace cloudbees-ci \
               -f values.yaml▼

After installing CloudBees CI on modern cloud platforms, you should:

Create a DNS record so that you can access the CloudBees CI on modern cloud platforms deployment.
Verify the installation of your CloudBees CI on modern cloud platforms cluster.
Sign-in to your CloudBees CI on modern cloud platforms cluster in FIPS-mode to see it in action.