Install CloudBees CI on Amazon Elastic Kubernetes Service

6 minute read

The following steps describe a setup using HTTPS with a domain name on Amazon Elastic Kubernetes Service (EKS).

Prerequisites

These prerequisites require access to AWS with administrative privileges.

  1. Install a Gateway API controller in your EKS cluster. Any conformant Gateway API implementation is supported. For tested implementations, refer to Kubernetes Gateway API supported implementations.

    Complete all Gateway API prerequisites before proceeding.

  2. Configure TLS certificates for your domain. For subdomain-based routing, use a wildcard certificate (for example, *.cloudbees-core.example.com).

    Configure the certificate based on where TLS termination occurs:

    • Load balancer-level TLS termination: Create a certificate in AWS Certificate Manager (ACM). The Gateway listener uses the HTTP protocol and does not reference a TLS Secret.

    • Gateway-level TLS termination: Store the certificate as a kubernetes.io/tls Secret in the Gateway namespace. Refer to Verify TLS certificate for details.

  3. Configure DNS to resolve to the Gateway external address. For subdomain-based routing, create a wildcard DNS record (for example, *.cloudbees-core.example.com) pointing to the Gateway external address. When using Route 53, create an alias record. Optionally, set up External-DNS to automate DNS record management.

Configure your environment

Complete the following steps to configure your environment for the CloudBees CI on modern cloud platforms installation. You must have access to the EKS cluster.

CloudBees CI on modern cloud platforms uses Docker containers to run a cluster of computers within the Kubernetes container management system. You can optionally verify the CloudBees CI on modern cloud platforms Docker images to ensure their origin and authenticity.

  1. Add the CloudBees Helm chart repository as follows:

    helm repo add cloudbees https://public-charts.artifacts.cloudbees.com/repository/public/ helm repo update
  2. Before initiating an installation, type the following commands to create a namespace:

    kubectl create namespace cloudbees-core kubectl config set-context $(kubectl config current-context) --namespace=cloudbees-core
    If you are using Kubernetes Gateway API for routing, also run kubectl label namespace cloudbees-core cloudbees.com/gateway-routes=enabled to allow HTTPRoutes to attach to the Gateway.
  3. Create a values.yaml file with the following content:

    OperationsCenter:
      HostName: {domain-name}(1)
      Protocol: https(2)
    Gateway:
      Enabled: true(3)
      Name: "<gateway-name>"(4)
      Namespace: "<gateway-namespace>"(5)
      # SectionName: "https"(6)
    Agents:
      SeparateNamespace:
        Enabled: true|false(7)
        Create: true|false(8)
    1 Replace {domain-name} with your domain name. Operations center is accessible from https://cjoc.{domain-name}.
    2 Set to http for development environments without TLS.
    3 Enables Gateway API support. The chart creates HTTPRoute resources instead of Ingress.
    4 Replace <gateway-name> with the name of the Gateway resource to reference in HTTPRoute resources.
    5 Replace <gateway-namespace> with the namespace where the Gateway resource is deployed.
    6 Optional. Specify when the Gateway has multiple listeners, and you need to target a specific one by name.
    7 Agents.SeparateNamespace.Enabled must have a value. If it is omitted, the installation or upgrade fails. CloudBees strongly recommends that you use a separate namespace for agents (true), to prevent pods from accessing sensitive information on the operations center. If separate namespaces are not used, an administrative monitor is displayed in CloudBees CI.
    8 Agents.SeparateNamespace.Create is optional. To create a separate namespace for agents, you can either allow Helm to create it automatically by adding the Agents.SeparateNamespace.Create=true argument, or create the namespace manually and label it with cloudbees.com/role: agents.

    For the full list of Helm values and installation options, refer to Install CloudBees CI on modern cloud platforms with Kubernetes Gateway API.

    If you plan to enable services, such as the CasC Controller Bundle Service, using this installation command, you must add the correct properties to the values.yaml file.

    For example, to enable the CasC Controller Bundle Service, you must add the following:

    CasCBundleService: enabled: true

    Refer to different CloudBees CI setups and Set up a managed controller using the CasC Controller Bundle Service for more information on where and how to enable services, such as the CasC Controller Bundle Service, in your CloudBees CI installation.

  4. You can optionally configure AWS Elastic Container Registry (ECR) to work with your AWS CloudBees CI on modern cloud platforms deployment. As long as your AWS cluster can access ECR, you can deploy CloudBees CI on modern cloud platforms from the ECR registry. Select one of the following:

Use AWS Elastic Container Registry (ECR) with CloudBees CI

You can deploy CloudBees CI itself from your ECR registry, as long as your cluster has access to ECR. To deploy CloudBees CI from your ECR registry, you must update the Helm chart so that it pulls CloudBees CI images from ECR instead of the public Docker registry.

CloudBees CI relies on the Jenkins Kubernetes Plugin to run Docker images in your Kubernetes cluster. The Jenkins Kubernetes Plugin documentation explains how to specify which Docker images are used in your Pipelines.

As part of running CloudBees CI on AWS, your Pipelines will run on AWS EC2 instances. AWS EC2 instances are automatically authenticated and authorized to use ECR, as long as the IAM profile used on the nodes allows access to ECR. For other private registries, the Jenkins Kubernetes Plugin uses a Kubernetes Secret imagePullSecret with the registry login credentials for the registry.

To configure AWS ECR to work with your CloudBees CI on modern cloud platforms deployment, complete the following steps.

  1. Create a registry in ECR. You can do this from the AWS Console or the AWS SDK. Once you create your registry, take note of your registry hostname, which will be formatted like this:

    # {id-number}.dkr.ecr.{region}.amazonaws.com 123456789012.dkr.ecr.us-east-1.amazonaws.com
  2. Run the following command to obtain a list of the CloudBees CI Docker images included in the Helm chart:

    helm show values cloudbees/cloudbees-core --jsonpath="{..Image.dockerImage}"

    Here’s an example of the output for version 2.176.1.4.

    $ cloudbees/cloudbees-cloud-core-oc:2.176.1.4 $ cloudbees/cloudbees-core-mm:2.176.1.4 $ cloudbees/cloudbees-core-agent:2.176.1.4 $ cloudbees/managed-master-hibernation-monitor:{version-number}
    The hibernation monitor version number differs from the version number for CloudBees CI.
  3. Pull the CloudBees CI images from Docker Hub

    To continue with the previous example, you would use the following commands to pull the CloudBees CI images for version 2.176.1.4.

    $ docker pull cloudbees/cloudbees-cloud-core-oc:2.176.1.4 $ docker pull cloudbees/cloudbees-core-mm:2.176.1.4 $ docker pull cloudbees/cloudbees-core-agent:2.176.1.4 $ docker pull cloudbees/managed-master-hibernation-monitor:{version-number}
  4. Run the following commands to tag the images. When you tag them you must replace cloudbees with your registry hostname. For example:

    $ docker tag cloudbees/cloudbees-cloud-core-oc:2.176.1.4 123456789012.dkr.ecr.us-east-1.amazonaws.com/cloudbees/cloudbees-cloud-core-oc:2.176.1.4 $ docker tag cloudbees/cloudbees-core-mm:2.176.1.4 123456789012.dkr.ecr.us-east-1.amazonaws.com/cloudbees/cloudbees-core-mm:2.176.1.4 $ docker tag cloudbees/cloudbees-core-agent:2.176.1.4 123456789012.dkr.ecr.us-east-1.amazonaws.com/cloudbees/cloudbees-core-agent:2.176.1.4 $ docker tag cloudbees/managed-master-hibernation-monitor:{version-number} 123456789012.dkr.ecr.us-east-1.amazonaws.com/cloudbees/managed-master-hibernation-monitor:{version-number}
  5. Run the following commands to push the images to ECR. You must replace the values below with the specific names of your images.

    $ docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/cloudbees/cloudbees-cloud-core-oc:2.176.1.4 $ docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/cloudbees/cloudbees-core-mm:2.176.1.4 $ docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/cloudbees/cloudbees-core-agent:2.176.1.4 $ docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/cloudbees/managed-master-hibernation-monitor:{version-number}
  6. Add the following Helm values to the values.yaml file, replacing 123456789012.dkr.ecr.us-east-1.amazonaws.com with the actual host name.

    Common: image: registry: 123456789012.dkr.ecr.us-east-1.amazonaws.com/cloudbees

Install CloudBees CI on modern cloud platforms

Type the following command to complete the installation:

helm install {release} \ {chart} \ --namespace {namespace} \ -f values.yaml

After you have installed CloudBees CI on modern cloud platforms, you should create a DNS record so that you can access the CloudBees CI on modern cloud platforms deployment.

Create a DNS record

Retrieve the Gateway external address:

kubectl get gateway <gateway-name> -n <gateway-namespace> -o jsonpath='{.status.addresses[0].value}'

Create a CNAME (or an alias in Route 53) wildcard DNS record (for example, *.cloudbees-core.example.com) pointing to this address.

For more information, refer to Verify DNS resolution.

Verify your CloudBees CI installation

The helm status command displays the status of the CloudBees CI release. This command is also executed after helm install and helm upgrade commands are executed.

The helm status command displays two areas of information:

  • Current CloudBees CI release

  • Installation notes

helm status cloudbees-core
NAME: cloudbees-core LAST DEPLOYED: Tue Apr 16 17:44:12 2019 NAMESPACE: cloudbees-core STATUS: DEPLOYED REVISION: 1 TEST SUITE: None NOTES: 1. Once operations center is up and running, get your initial admin user password by running: kubectl rollout status sts cjoc --namespace cloudbees-core kubectl exec cjoc-0 --namespace cloudbees-core -- cat /var/jenkins_home/secrets/initialAdminPassword 2. Visit https://cloudbees-core.example.com/cjoc/ 3. Sign in with the password from step 1.

Sign in to your CloudBees CI installation

Now that you’ve installed CloudBees CI and operations center, you’ll want to see your system in action.

  1. Type the following command to retrieve your administrative user password:

    kubectl exec cjoc-0 -- cat /var/jenkins_home/secrets/initialAdminPassword
  2. Open a browser to http://cloudbees-core.example.com/cjoc/.

  3. Sign in with the username admin and the password you retrieved.

For more information on running CloudBees CI on Kubernetes, refer to the CloudBees CI on modern cloud platforms administration guide.

Security considerations for CloudBees CI on modern cloud platforms

Once you have installed CloudBees CI on modern cloud platforms, CloudBees recommends that you fully review, understand, and implement the security measures provided in Trust model for CloudBees CI on modern cloud platforms.

In August 2020, the Jenkins project voted to replace the term master with controller. We have taken a pragmatic approach to cleaning these up, ensuring the least amount of downstream impact as possible. CloudBees is committed to ensuring a culture and environment of inclusiveness and acceptance - this includes ensuring the changes are not just cosmetic ones, but pervasive. As this change happens, please note that the term master has been replaced through the latest versions of the CloudBees documentation with controller (as in managed controller, client controller, team controller) except when still used in the UI or in code.