Installing CloudBees CI on modern cloud platforms on EKS

4 minute read

The following steps describe a public setup using HTTPS with a domain name.

Prerequisites

These prerequisites require access to AWS with administrative privileges as they involve IAM work.

  1. Follow the steps from AWS Load Balancer Controller to install the AWS Load Balancer controller in the EKS cluster.

    Since CloudBees CI on modern cloud platforms 2.277.4.2, networking/v1 Ingress is used when supported. Consequently, if you use EKS version 1.19 or later, the AWS Load Balancer Controller 2.2.0 or later is required.
  2. Create a certificate in AWS Certificate Manager for the required domain name. For cloudbees-core.example.com, we recommend creating a certificate for *.cloudbees-core.example.com when setting up CloudBees CI on modern cloud platforms using Subdomain: true.

  3. Set up External-DNS. Read the tutorial to set it up on AWS.

Configuring your environment

Complete the following steps to configure your environment for the CloudBees CI on modern cloud platforms installation. You must have access to the EKS cluster.

  1. Add the CloudBees Helm chart repository as follows:

    helm repo add cloudbees https://charts.cloudbees.com/public/cloudbees
    helm repo update
  2. Before initiating an installation, type the following commands to create a namespace:

    kubectl create namespace cloudbees-core
    kubectl config set-context $(kubectl config current-context) --namespace=cloudbees-core
  3. Create a file named values.yaml with the following content.

    Subdomain: true
    OperationsCenter:
      HostName: cloudbees-core.example.com (1)
      Protocol: https (2)
      Platform: eks
      Ingress:
        Class: alb
        Annotations:
          alb.ingress.kubernetes.io/scheme: internet-facing (3)
    1Replace cloudbees-core.example.com with your domain name. operations center will be accessible from https://cjoc.cloudbees-core.example.com.
    2Remove to use http
    3Can be omitted in order to use an internal load balancer.

    Refer to the full list of annotations, if more details are needed.

  4. You can optionally configure AWS Elastic Container Registry (ECR) to work with your AWS CloudBees CI on modern cloud platforms deployment. As long as your AWS cluster can access ECR, you can deploy CloudBees CI on modern cloud platforms from the ECR registry. Select one of the following:

Using AWS Elastic Container Registry (ECR) with CloudBees CI

You can deploy CloudBees CI itself from your ECR registry, as long as your cluster has access to ECR. To deploy CloudBees CI from your ECR registry, you must update the Helm chart so that it pulls CloudBees CI images from ECR instead of the public Docker registry.

CloudBees CI relies on the Jenkins Kubernetes Plugin to run Docker images in your Kubernetes cluster. The Jenkins Kubernetes Plugin documentation explains how to specify which Docker images are used in your Pipelines.

As part of running CloudBees CI on AWS, your Pipelines will run on AWS EC2 instances. AWS EC2 instances are automatically authenticated and authorized to use ECR, as long as the IAM profile used on the nodes allows access to ECR. For other private registries, the Jenkins Kubernetes Plugin uses a Kubernetes Secret imagePullSecret with the registry login credentials for the registry.

To configure AWS ECR to work with your CloudBees CI on modern cloud platforms deployment, complete the following steps.

  1. Create a registry in ECR. You can do this from the AWS Console or the AWS SDK. Once you create your registry, take note of your registry hostname, which will be formatted like this:

    # {id-number}.dkr.ecr.{region}.amazonaws.com
    123456789012.dkr.ecr.us-east-1.amazonaws.com
  2. Run the following command to obtain a list of the CloudBees CI Docker images included in the Helm chart:

    helm show values cloudbees/cloudbees-core --jsonpath="{..Image.dockerImage}"

    Here’s an example of the output for version 2.176.1.4.

    $ cloudbees/cloudbees-cloud-core-oc:2.176.1.4
    $ cloudbees/cloudbees-core-mm:2.176.1.4
    $ cloudbees/cloudbees-core-agent:2.176.1.4
    $ cloudbees/managed-master-hibernation-monitor:{version-number}
    The hibernation monitor version number differs from the version number for CloudBees CI.
  3. Pull the CloudBees CI images from Docker Hub

    To continue with the previous example, you would use the following commands to pull the CloudBees CI images for version 2.176.1.4.

    $ docker pull cloudbees/cloudbees-cloud-core-oc:2.176.1.4
    $ docker pull cloudbees/cloudbees-core-mm:2.176.1.4
    $ docker pull cloudbees/cloudbees-core-agent:2.176.1.4
    $ docker pull cloudbees/managed-master-hibernation-monitor:{version-number}
  4. Run the following commands to tag the images. When you tag them you must replace cloudbees with your registry hostname. For example:

    $ docker tag cloudbees/cloudbees-cloud-core-oc:2.176.1.4  123456789012.dkr.ecr.us-east-1.amazonaws.com/cloudbees-cloud-core-oc:2.176.1.4
    $ docker tag cloudbees/cloudbees-core-mm:2.176.1.4  123456789012.dkr.ecr.us-east-1.amazonaws.com/cloudbees-core-mm:2.176.1.4
    $ docker tag cloudbees/managed-master-hibernation-monitor:{version-number} 123456789012.dkr.ecr.us-east-1.amazonaws.com/cloudbees/managed-master-hibernation-monitor:{version-number}
    $ docker tag cloudbees/cloudbees-core-agent:2.176.1.4 123456789012.dkr.ecr.us-east-1.amazonaws.com/cloudbees-core-agent:2.176.1.4
  5. Run the following commands to push the images to ECR. You must replace the values below with the specific names of your images.

    $ docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/cloudbees-cloud-core-oc:2.176.1.4
    $ docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/cloudbees-core-mm:2.176.1.4
    $ docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/cloudbees/managed-master-hibernation-monitor:{version-number}
    $ docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/cloudbees-core-agent:2.176.1.4
  6. Add the following Helm values to the values.yaml file. You must replace the values below with the specific names of your images.

    OperationsCenter:
      Image:
        dockerImage: 123456789012.dkr.ecr.us-east-1.amazonaws.com/cloudbees-core-oc:2.176.1.4
    Master:
      Image:
        dockerImage: 123456789012.dkr.ecr.us-east-1.amazonaws.com/cloudbees-core-mm:2.176.1.4
    Hibernation:
      Image:
        dockerImage: 123456789012.dkr.ecr.us-east-1.amazonaws.com/cloudbees/managed-master-hibernation-monitor:{version-number}
    Agent:
      Image:
        dockerImage: 123456789012.dkr.ecr.us-east-1.amazonaws.com/cloudbees-core-agent:2.176.1.4

Installing CloudBees CI on modern cloud platforms

Type the following command to complete the installation:

helm upgrade --install {release} \
               {chart} \
               --namespace {namespace} \
               -f values.yaml

After you have installed CloudBees CI on modern cloud platforms, you should create a DNS record so that you can access the CloudBees CI on modern cloud platforms deployment.

Creating a DNS record

Look up the ALB host name using the following command

kubectl --namespace {namespace} get ingress cjoc -o go-template='{{ range .status.loadBalancer.ingress }}{{ .hostname}}{{"\n"}}{{end}}'

This should return something like k8s-cloudbees-aabbccddee-1234567890.<region>.elb.amazonaws.com. Create a CNAME (or an alias in route 53) DNS record targeting this name in order to access the CloudBees CI on modern cloud platforms deployment.

Verifying your CloudBees CI installation

The helm status command displays the status of the CloudBees CI release. This command is also executed after helm install and helm upgrade commands are executed.

The helm status command displays two areas of information:

  • Current CloudBees CI release

  • Installation notes

helm status cloudbees-core
NAME:   cloudbees-core
LAST DEPLOYED: Tue Apr 16 17:44:12 2019
NAMESPACE: cloudbees-core
STATUS: DEPLOYED
REVISION: 1
TEST SUITE: None
NOTES:
1. Once operations center is up and running, get your initial admin user password by running:
  kubectl rollout status sts cjoc --namespace cloudbees-core
  kubectl exec cjoc-0 --namespace cloudbees-core -- cat /var/jenkins_home/secrets/initialAdminPassword
2. Visit https://cloudbees-core.example.com/cjoc/

3. Sign in with the password from step 1.

Signing in to your CloudBees CI installation

Now that you’ve installed CloudBees CI and operations center, you’ll want to see your system in action.

  1. Type the following command to retrieve your administrative user password:

    kubectl exec cjoc-0 -- cat /var/jenkins_home/secrets/initialAdminPassword
  2. Open a browser to http://cloudbees-core.example.com/cjoc/.

  3. Sign in with the username admin and the password you retrieved.

For more information on running CloudBees CI on Kubernetes, see the CloudBees CI on modern cloud platforms administration guide.