Issue
When creating a new Managed controller, some manual configuration is necessary to be able to connect an external agent. Is there a way to automate this Managed controller creation when using an Helm setup?
Environment
-
CloudBees CI (CloudBees Core) on modern cloud platforms - Managed controller
-
Helm based installation (Helm version 2 or 3)
Resolution
Prerequisite:
-
Ensure you replace
cloudbees-core
by the namespace used by your ingress-controller
The following bash script will:
-
Take a backup of your current configuration and save it under
backup.yaml
(note, you can also rollback with helm). -
Create the new rules and save them under
rules.yaml
. -
Apply the newly created rules to the installation.
Important notes
This script will update several Kubernetes objects in your cluster. From a Kubernetes point of view, thanks to the mechanism of Rolling Updates, there should not be any service interruptions. However, the script also updates nginx configuration, which means that most likely the nginx service will be restarted. Therefore all active connections to the cluster might get dropped (eg an external agent connected to the controller). We therefore recommend this script to run only at a quiet time.
The script is provided AS IS and you should review what it is doing before applying it.
Type of rules generated
The script is generating rules for a backing Load Balancer using the proxy protocol (like an AWS ELB).
If your Load Balancer is not using the proxy protocol, you should modify the rule creation to not append :PROXY
This means that:
ROUTES=$(kubectl get svc -l com.cloudbees.cje.type=master --all-namespaces -o jsonpath="{range .items[*]}{' "'}{.spec.ports[?(@.name=='agent')].port}{'": "'}{.metadata.namespace}{'/'}{.metadata.name}{':'}{.spec.ports[?(@.name=='agent')].port}{':PROXY"\n'}\{end}")
becomes:
ROUTES=$(kubectl get svc -l com.cloudbees.cje.type=master --all-namespaces -o jsonpath="{range .items[*]}{' \"'}{.spec.ports[?(@.name=='agent')].port}{'\": \"'}{.metadata.namespace}{'/'}{.metadata.name}{':'}{.spec.ports[?(@.name=='agent')].port}{'\"\n'}{end}")
If you are using Helm 2
Prerequisites
-
Helm 2
-
kubectl configured to connect to the Kubernetes Cluster containing your CloudBees Core installation
-
jq
Script Parameters
You need to adjust the RELEASE_NAME
to your current release name.
Run the script once as a dry run (no changes actually applied).
Make sure the output is the expected one.
Then update DRY_RUN
to false.
Script
#!/bin/bash set -euo pipefail == Prerequisites of this script == * kubectl configured to connect to the Kubernetes Cluster containing your CloudBees Core installation == * helm 2 == * jq == Script will == 1. Take a backup of your current configuration and save it under backup.yaml == 2. Create the new rules and save them under rules.yaml == 3. Apply the newly created rules to the installation == Parameters == * RELEASE_NAME: the name of the helm release for the CloudBees Core installation, defaults to cloudbees-core. == You can find this value by running helm ls == * DRY_RUN: set to something else than true to run the actual update, defaults to true. == BEGIN PARAMETERS RELEASE_NAME='cloudbees-core' DRY_RUN=false == END PARAMETERS printf 'Backing up the current configuration for configuration %s under backup.yaml\n' "$\{RELEASE_NAME}" helm get values "$\{RELEASE_NAME}" >backup.yaml printf 'Done\n' printf 'Creating the rules\n' ROUTES=$(kubectl get svc -l com.cloudbees.cje.type=master --all-namespaces -o jsonpath="{range .items[*]}{' "'}{.spec.ports[?(@.name=='agent')].port}{'": "'}{.metadata.namespace}{'/'}{.metadata.name}{':'}{.spec.ports[?(@.name=='agent')].port}{':PROXY"\n'}\{end}") cat >rules.yaml <<EOF nginx-ingress: tcp: $\{ROUTES} EOF printf 'Done\n' printf 'Will write the following rules to your ingress controller:\n%s\n' "$(cat rules.yaml)" [cols=2*] |=== | CHART_NAME=$(helm list "$\{RELEASE_NAME}" --output json | jq -j '.Releases[0].Chart') |=== printf "Found the following chart to update %s\n" "$\{CHART_NAME}" if [ "$\{DRY_RUN}" = true ] ; then helm upgrade --dry-run --debug --version $(sed "s/$\{RELEASE_NAME}-//" <<< "$\{CHART_NAME}") --description 'Ingress routes generation' --values backup.yaml --values rules.yaml "$\{RELEASE_NAME}" cloudbees/cloudbees-core else helm upgrade --version $(sed "s/$\{RELEASE_NAME}-//" <<< "$\{CHART_NAME}") --description 'Ingress routes generation' --values backup.yaml --values rules.yaml "$\{RELEASE_NAME}" cloudbees/cloudbees-core fi
If you are using Helm 3
Prerequisites
-
Helm 3
-
kubectl configured to connect to the Kubernetes Cluster containing your CloudBees Core installation
-
jq
Script Parameters
You need to adjust the RELEASE_NAME
to your current release name.
Run the script once as a dry run (no changes actually applied).
Make sure the output is the expected one.
Then update DRY_RUN
to false.
Script
#!/bin/bash set -euo pipefail # Prerequisites of this script: # * kubectl configured to connect to the Kubernetes Cluster containing your CloudBees Core installation # * helm 3. It is assumed the CloudBees repository is named cloudbees (ie you added it with helm repo add cloudbees https://public-charts.artifacts.cloudbees.com/repository/public/ # * jq # Script will: # 1. Take a backup of your current configuration and save it under backup.yaml # 2. Create the new rules and save them under routes.yaml # 3. Apply the newly created rules to the installation # Parameters # * RELEASE_NAME: the name of the helm release for the CloudBees Core installation, defaults to cloudbees-core. # You can find this value by running helm ls # * DRY_RUN: set to something else than true to run the actual update, defaults to true. # BEGIN PARAMETERS RELEASE_NAME='cloudbees-core' DRY_RUN=true # END PARAMETERS printf 'Backing up the current configuration for configuration %s under backup.yaml\n' "${RELEASE_NAME}" helm get values "${RELEASE_NAME}" >backup.yaml printf 'Done\n' printf 'Creating the rules\n' ROUTES=$(kubectl get svc -l com.cloudbees.cje.type=controller --all-namespaces -o jsonpath="{range .items[*]}{' \"'}{.spec.ports[?(@.name=='agent')].port}{'\": \"'}{.metadata.namespace}{'/'}{.metadata.name}{':'}{.spec.ports[?(@.name=='agent')].port}{':PROXY\"\n'}{end}") cat >rules.yaml <<EOF nginx-ingress: tcp: ${ROUTES} EOF printf 'Done\n' printf 'Will write the following rules to your ingress controller:\n%s\n' "$(cat rules.yaml)" CHART_NAME=$(helm list --filter "${RELEASE_NAME}" --output json | jq -j '.[0].chart') printf "Found the following chart to update %s\n" "${CHART_NAME}" if [ "${DRY_RUN}" = true ] ; then helm upgrade --dry-run --debug --version $(sed "s/${RELEASE_NAME}-//" <<< "${CHART_NAME}") --values backup.yaml --values rules.yaml "${RELEASE_NAME}" cloudbees/cloudbees-core else helm upgrade --version $(sed "s/${RELEASE_NAME}-//" <<< "${CHART_NAME}") --values backup.yaml --values rules.yaml "${RELEASE_NAME}" cloudbees/cloudbees-core fi
Tested product/plugin versions
-
CloudBees CI (CloudBees Core) 2.204.1.3
-
Helm 2.16.1 and Helm 3.0.1