How to configure CloudBees CD/RO for very large Helm deployments
A very large Helm deployment of CloudBees CD/RO is:
-
~ 100K jobs per day
-
~ 2000 running pipelines per day
-
~ 5M job steps per day
Active customers and partners can refer to the CloudBees CD/RO Level 2: Helm for Very Large Kubernetes Deployments training course for details.
The following tables summarize the changes to the default Helm chart for very large deployments:
CloudBees CD/RO server values
For the default CloudBees CD/RO Helm charts, refer to CloudBees CD/RO server values. You can find these cloudbees-flow
values referenced in the values.yaml
under the Flow server configuration section.
Name | Value |
---|---|
server.replicas |
4 |
server.ecconfigure |
"--serverInitMemory=70 --serverMaxMemory=85" |
server.resources.limits.cpu |
16 |
server.resources.limits.memory |
128Gi |
server.resources.requests.cpu |
4 |
server.resources.requests.memory |
16Gi |
CloudBees CD/RO web server values
For the default CloudBees CD/RO Helm charts, refer to CloudBees CD/RO web server values. You can find these cloudbees-flow
values referenced in the values.yaml
under the Flow web server configuration section.
Name | Value |
---|---|
web.replicas |
2 |
Repository values
For the default CloudBees CD/RO Helm charts, refer to Repository values. You can find these cloudbees-flow
values referenced in the values.yaml
under the Flow repository configuration section.
Name | Value |
---|---|
repository.resources.requests.memory |
768Mi |
CloudBees Analytics server values
For the default CloudBees CD/RO Helm charts, refer to CloudBees Analytics server values. You can find these cloudbees-flow
values referenced in the values.yaml
under the Analytics server configuration section.
Name | Value |
---|---|
analytics.replicas |
|
analytics.esMinMasterNodes |
|
analytics.heapSize |
|
analytics.resources.limits.cpu |
|
analytics.resources.limits.memory |
|
analytics.resources.requests.cpu |
|
analytics.resources.requests.memory |
|
CloudBees CD/RO agent values
For the default CloudBees CD/RO Helm charts, refer to CloudBees CD/RO agent configuration values You can find these cloudbees-flow-agent
values referenced in the agent values.yaml
.
Name | Value |
---|---|
replicas |
|
How to set global values in CloudBees CD/RO Helm charts
Starting with v2023.10.0, you can configure global values for image settings to use for all CloudBees CD/RO components. In v2023.10.0 and later cloudbees-flow
values files, you can find these configurations by searching for Global configurations section
.
When you set a global image, it is applied to the following components:
-
server
-
web
-
repository
-
analytics
-
analytics.backup
-
boundAgent
-
zookeeper
-
internalGatewayAgent
-
externalGatewayAgent
This option improves usability and enables you to set and update chart values for these components in a single location, instead of for each component individually. However, this setting is optional, and you can still configure CloudBees CD/RO components individually to pull specific images.
The following sections do not apply values from the global section and must be individually configured:
For information on setting custom images, refer to How to configure custom images for CloudBees CD/RO. |
Setting global chart values
Setting global values for image settings is an optional configuration for CloudBees CD/RO v2023.10.0 and later, meant to improve usability and maintainability of your Helm charts. Global chart values allow you to set a value in one location that, where applicable, is applied to all CloudBees CD/RO components.
Global-level chart values override component-level chart values. This means, if you have both a global-level value set and component-level value set, the global value is taken. Before setting any global-level value, CloudBees strongly suggests to ensure the global value is valid and correct for all instances where it will be applied. |
The following is an example of setting a global image value:
-
In your v2023.10.0 and later values file, navigate to the Global configurations section.
-
Provide values for:
-
global.cloudbees.imageRegistry
-
global.cloudbees.imageTag
-
(OPTIONAL)
global.cloudbees.imagePullSecrets
You must enable
imagePullSecrets
if you are using a private registry. Secrets must be manually created or already exist in the namespace.
-
-
(OPTIONAL) For CloudBees CD/RO component values that have a
global.<value>
set, remove the<component>.<value>
.Global-level configurations override component-level configurations. However, from a troubleshooting point of view, having two different values configured in different locations in your values file may cause some confusion. Because of this, CloudBees suggests removing the component-level configuration for values already set at the global level.
You are now able to update the global image to use for all CloudBees CD/RO components in a single location in your values file.
How to configure custom resources for init job
If using CloudBees CD/RO v2024.06.0 or later, you can configure the resources allocated to the CloudBees CD/RO init job
within your values file using jobInit.resources
. To get started:
-
If you do not already have a v2024.06.0 or later values file, update your existing values file with the following fields for
jobInit
:resources: limits: cpu: 4 memory: 6Gi requests: cpu: 2 memory: 6Gi
-
Set your custom values for the fields.
The default values are the minimum suggested values. For some environments these settings may not be sufficient, and result in the init job
failing. If this occurs, increase the default settings for your environment, and run the installation or upgrade again. -
Ensure the YAML is valid, and save your changes.
-
Deploy the updated chart to your environment using your
helm upgrade
command.
The CloudBees CD/RO init job
will now be allocated the configured resources.
How to configure custom images for CloudBees CD/RO
If you need to use custom images for individual components in your project, you can configure the image information in your myvalues.yaml
. The following sections describe how to set custom images for:
If you set a Before configuring custom images for bound agents, ensure you do not have images configured for |
Configure custom bound agent images
This section is intended for use with v2023.10.0 and later Helm charts. For CloudBees CD/RO Helm charts v2023.08.0 and earlier, the boundAgent included image configuration fields in the Helm chart by default.
|
Add a custom boundAgent
image to Helm chart
In CloudBees CD/RO v2023.10.0 and above Helm charts, global image configurations were implemented in the cloudbees-flow
values file, values.global.cloudbees
. As part of this change, bound agents were reconfigured to use the cloudbees-flow-agent.values.image
values. Additionally, the boundAgent.imageRegistry
values were removed from the cloudbees-flow
values file.
If your project uses custom images for bound agents, you must make the following changes to CloudBees CD/RO v2023.10.0 and above Helm charts before starting the upgrade:
-
In your
cloudbees-flow
myvalues.yaml
, ensure novalues.global.cloudbees
image value is set.If you set both a values.global.cloudbees
image value and any component-level image values in yourmyvalues.yaml
, thevalues.global.cloudbees
image value overrides all component-level image values. -
In the
boundAgent
section of yourcloudbees-flow
values file, add the following chart configuration:images: ## Image registry to pull CloudBees CD/RO boundAgent images from. ## Example: registry: "123456789012.dkr.ecr.us-east-1.amazonaws.com" registry: "<YOUR-IMAGE-REGISTRY>" ## The `imageRepository` in the `images.registry` to pull the bound agent image from. imageRepository: "<YOUR-IMAGE-REPO>" ## CloudBees CD/RO bound agent image tag to pull. tag: "<YOUR-IMAGE-TAG>"
-
For example, if your project uses an internal registry called
internal.registry.example/cloudbees
::images: ## Image registry to pull CloudBees CD/RO boundAgent images from. ## Example: registry: "123456789012.dkr.ecr.us-east-1.amazonaws.com" registry: "internal.registry.example/cloudbees" ## The `imageRepository` in the `images.registry` to pull the bound agent image from. imageRepository: "cbflow-boundagent" ## CloudBees CD/RO bound agent image tag to pull. tag: "1.0"
-
You can now upgrade to CloudBees CD/RO v2023.10.0 and above using your custom bound agent image.
Configure custom ingress-nginx images
Add a custom ingress-nginx
image to Helm chart
If your project uses custom images for ingress-nginx, you must make the following image changes to the CloudBees CD/RO Helm charts in your myvalues.yaml
before starting the upgrade.
In the ingress-nginx
section of your cloudbees-flow
values file, add the following chart configuration:
controller: image: registry: <YOUR-IMAGE-REGISTRY> image: <YOUR-IMAGE> ## For backward compatibility, consider setting the full image URL via the repository value below. ## Use *either* the current default registry/image or the repository format. Otherwise, installing the chart by providing the `values.yaml` will fail. # repository: tag: "<YOUR-IMAGE-TAG>" digest: <YOUR-DIGEST-HASH> admissionWebhooks: patch: registry: <YOUR-IMAGE-REGISTRY> image: <YOUR-IMAGE> ## For backward compatibility, consider setting the full image URL via the repository value below. ## Use *either* the current default registry/image or the repository format. Otherwise, installing the chart by providing the `values.yaml` will fail. # repository: tag: "<YOUR-IMAGE-TAG>" digest: <YOUR-DIGEST-HASH> ## Set images if enabled defaultBackend: enabled: false image: registry: <YOUR-IMAGE-REGISTRY> image: <YOUR-IMAGE> ## For backward compatibility, consider setting the full image URL via the repository value below. ## Use *either* the current default registry/image or the repository format or installing the chart by providing the `values.yaml` will fail. # repository: tag: "<YOUR-IMAGE-TAG>"
For example, if your project uses an internal registry called internal.registry.example/ingress-nginx
:
controller: image: registry: internal.registry.example/ingress-nginx image: controller ## For backward compatibility, consider setting the full image URL via the repository value below. ## Use *either* the current default registry/image or the repository format or installing the chart by providing the `values.yaml` will fail. # repository: tag: "v1.1.0" digest: sha256:f766669fdcf3dc26347ed273a55e754b427eb4411ee075a53f30718b4499076a admissionWebhooks: patch: registry: internal.registry.example/ingress-nginx image: kube-webhook-certgen ## For backward compatibility, consider setting the full image URL via the repository value below. ## Use *either* the current default registry/image or the repository format or installing the chart by providing the `values.yaml` will fail. # repository: tag: "v1.1.1" digest: sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660 ## Set images if enabled defaultBackend: enabled: false image: registry: internal.registry.example/ingress-nginx image: defaultbackend-am