Agent configuration

This page covers configuration articles related to CloudBees CD/RO agents. Agents are worker processes that execute tasks such as running jobs, deploying applications, and executing procedures. When running CloudBees CD/RO on Kubernetes, you can configure agents in various ways to support different deployment topologies and operational requirements. This page covers essential agent configuration tasks including setting up gateway agents for hybrid deployments, configuring resource names and workspaces, managing SSL/TLS certificates, and optimizing agent pod behavior.

Configure internal and external agents

CloudBees CD/RO installation on Kubernetes uses internal services within the cluster to communicate between components, which ensures that communication within the cluster is efficient and secure. A common method for deploying CloudBees CD/RO on Kubernetes is to install CloudBees CD/RO components inside the Kubernetes cluster while installing worker agents such as deployment targets outside the cluster, such as on VMs and traditional platforms.

Figure 1. Install CloudBees CD/RO components inside Kubernetes cluster

In a mixed inside and outside deployment use case, communications with the CloudBees CD/RO components happen with internal services installed on Kubernetes, while communication with CD agents happen outside the cluster. These external agents can communicate with the internal components using a pair of gateway agents.

Gateway agent pairs can be installed using one of these methods:

CloudBees CD/RO Helm chart
CloudBees CD/RO agent Helm chart

Install gateway agent pairs using the CloudBees CD/RO Helm chart

Enable both internalGatewayAgent and externalGatewayAgent to enable the CloudBees CD/RO chart to install the agent sub-chart as gateway agents. Make sure you set ingress.host and externalGatewayAgent.service.publicHostName both as the same value:
```
ingress:
  host: my-flow.example.com

internalGatewayAgent:
  enabled: true

externalGatewayAgent:
  enabled: true
  service:
    publicHostName: my-flow.example.com▼
```

Ensure your default values file has ingress-nginx TCP config map settings uncommented, to expose the 7800 gateway agents service:

ingress-nginx:
  tcp:
    8200: "{{ .Release.Namespace }}/flow-repository:8200"
    8443: "{{ .Release.Namespace }}/flow-server:8443"
    7800: "{{ .Release.Namespace }}/gateway-external-agent-flow-agents:7800"▼

If you are not creating an ingress controller as part of the CloudBees CD/RO chart because you already have an existing ingress controller make sure you have tcpconfig map added to expose above ports and services (for more details, refer to the Kubernetes documentation).
```
helm repo add cloudbees https://public-charts.artifacts.cloudbees.com/repository/public/
helm repo update
helm install releaseName cloudbees/cloudbees-flow \
          -f valuesFile --namespace nameSpace --timeout 10000s▼
```

Install gateway agents using the CloudBees CD/RO agent Helm chart

To create the gateway agent using the agent chart:

Create the server secret:

# Flow server credentials to use in order to register
# with agent as a resource on the Flow server.

kubectl create secret generic flow-agent-secret \
 --from-literal=CBF_SERVER_USER='admin' \
 --from-literal=CBF_SERVER_PASSWORD=<flow-server-password> \
 -n <namespace>▼

Create a values file for the internal agent called gateway-default-agent.yaml using the following command:

resourceName: gateway-default-agent
replicas: 1
flowCredentials:
  existingSecret: flow-agent-secret
service:
  publicHostName: gateway-default-agent-flow-agents▼

Install the gateway internal agent using gateway-default-agent.yaml:

helm install gateway-default-agent cloudbees/cloudbees-flow-agent \
-f gateway-default-agent.yaml -n <namespace>▼

Create a values file for the external agent called gateway-external-agent.yaml that adds zoneName, publicHostName and gateway configuration:

resourceName: gateway-external-agent
zoneName: external
service:
  # External DNS hostname that the external agents would use to communicate
  # with the external gateway agent
  publicHostName: my-flow.example.com

# configure gateway using external gateway agent
gateway:
  # configure gateway using this agent
  enabled: true
  # Name of the gateway to create
  name: external
  # Name of gateway agent to pair with as gateway resource 2
  pairedResourceName: gateway-default-agent

flowCredentials:
  existingSecret: flow-agent-secret▼

Install the gateway external agent using gateway-external-agent.yaml:

helm install gateway-external-agent cloudbees/cloudbees-flow-agent \
-f gateway-external-agent.yaml -n <namespace>▼

Autoscaling gateway agents

External gateway agents allow connectivity to be routed through gateways internally within your Kubernetes cluster or externally via an external endpoint. Typically, agents are included within the gateway configuration. As each pod replica is registered individually as a resource, and their gateway agents created 1:1, this creates a single point of failure within each replica.

This model is shown in the following example Kubernetes cluster using an external gateway agent:

Figure 2. External gateway agents

Although this model generally provides internal and external connectivity, it doesn’t promote the goal of high availability. However, horizontally autoscaling your gateway agents allows your cluster to dynamically handle workloads or the loss of a gateway agent. CloudBees CD/RO provides this option for horizontal pod autoscaling (HPA) with HPA gateway agents.

By default, the CloudBees CD/RO server does not include HPA gateway agents. You must manually configure gateway agent autoscaling.

Configuring HPA gateway agents

Using HPA gateway agents allows you to distribute the cluster workload over one or many agent replicas and provides availability if one fails. The following steps allow your HPA gateway agents to be registered as services, using a service name, instead of the resource name of individual pods.

The CloudBees CD/RO cloudbees-flow Helm chart is available in the public cloudbees-examples repository. CloudBees recommends you save this in a local myvalues.yaml to retain it in your environment.

In your CloudBees CD/RO cloudbees-flow Helm chart, add the following:

internalGatewayAgent:
    enabled: true
    autoscaling:
      enabled: true
      minReplicas: 1
      maxReplicas: 2
      targetCPUUtilizationPercentage: 80
      targetMemoryUtilizationPercentage: 80

# cloudbees-flow-agent chart configurations used for creating the
# external agent for the gateway
externalGatewayAgent:
    enabled: true
    autoscaling:
      enabled: true
      minReplicas: 1
      maxReplicas: 2
      targetCPUUtilizationPercentage: 80
      targetMemoryUtilizationPercentage: 80▼

For internalGatewayAgent, configure the following for your cluster:
1. minReplicas
2. maxReplicas
3. targetCPUUtilizationPercentage
4. targetMemoryUtilizationPercentage
  
  For HPA gateway agents to work, internalGatewayAgent.enabled and internalGatewayAgent.autoscaling.enabled must be set to true.
For externalGatewayAgent, configure the following for your cluster:
1. minReplicas
2. maxReplicas
3. targetCPUUtilizationPercentage
4. targetMemoryUtilizationPercentage
  
  For HPA gateway agents to work, externalGatewayAgent.enabled and externalGatewayAgent.autoscaling.enabled must be set to true.

Autoscaling non-gateway agents

CloudBees CD/RO does not support horizontal autoscaling of non-gateway agents. Attempts to follow these instructions for non-gateway agents will result in an error message for an invalid configuration.

Configure agent resource name templates

Generally, you set the name for the agent resource with the value agent.resourceName, but this only works when the agent deployment has only one replica. If there are multiple replicas, then each of them tries to register itself using the same resource name. As a result, only one replica from the entire deployment will be registered as a resource.

The resource name defaults to hostname, but other template values are available to fully qualify the resource:

Ordinary Helm template values. For example: {{ .Release.Name }}, {{ .Release.Namespace }}, and so on.
In addition, the following two special template values are allowed:
- {{ hostname }}: The actual resource hostname.
- {{ ordinalIndex }}: The serial replica index in the StatefulSet.

Example:

resourceName: "myResource - {{ .Release.Name }} - {{ ordinalIndex }}"

Configure existing secrets for bound agents

If you are using an existing secret in your values file for flowCredentials.existingSecret, you may have gotten the following error message when trying to install the bound agent (cbflow-agent):

To use flowCredentials.existingSecret, please set \
boundAgent.flowCredentials.existingSecret to the same value \
as flowCredentials.existingSecret in your values file.▼

This is because CloudBees CD/RO expects if you are using an existing secret in flowCredentials.existingSecret, the same existingSecret is used for the bound agent.

To use your existing CloudBees CD/RO flow-server secret for the bound agent (cbflow-agent):

Open your myvalues.yaml, and search for boundAgent:.

In the boundAgent configuration, add the following:

  flowCredentials:
    existingSecret: <your-flowCredentials.existingSecret-value>▼

Ensure the entries are correctly indented, and save your file.

On your next install or upgrade attempt, this should remove the boundAgent.flowCredentials.existingSecret error message.

Configure graceful termination period for agent pods

For various reasons, Kubernetes clusters may migrate agents pod across nodes, which is normally triggered by a sigterm signal to the current agent pod that immediately terminates it. To provide better monitoring and graceful termination options, support was added to the CloudBees CD/RO Helm charts to configure termination grace periods for agent pods, with a default grace period of 30 seconds.

Options to add graceful termination periods were added in CloudBees CD/RO v2024.03.0 and later. This option is not available in CloudBees CD/RO v2023.12.0 and earlier.

To configure graceful termination for agent pods:

Open your CloudBees CD/RO cloudbees-flow-agent v2024.03.0 or later my-values.yaml.
Search for terminationGracePeriodSeconds.
Set the desired graceful termination period in seconds

The default value is 30 seconds.
Update your Kubernetes CloudBees CD/RO installation to apply the new configuration.

Once the first agent is deployed with ReadWriteMany access mode, subsequent agents deployed for the same workspace with storage.volumes.agentWorkspace.existingClaim to true share the first agent’s workspace. The following example shows how to set up flow-agent-1 and flow-agent-2 to share the same workspace, MyWorkspace.

Deploy the first agent with storage.volumes.agentWorkspace.accessMode set to ReadWriteMany. This creates the persistent volume claim, setting up the scenario where agents can use the flow-agent-workspace shared workspace.

helm install flow-agent-1 cloudbees-flow-agent -f <valuesFile> \
--set storage.volumes.agentWorkspace.accessMode=ReadWriteMany \
--set storage.volumes.agentWorkspace.name=MyWorkspace \
--namespace <nameSpace> --timeout 10000▼

Deploy subsequent agents to the same workspace with storage.volumes.agentWorkspace.existingClaim to true.

helm install flow-agent-2 cloudbees-flow-agent -f <valuesFile>\
--set storage.volumes.agentWorkspace.existingClaim=true \
--set storage.volumes.agentWorkspace.name=MyWorkspace \
--namespace <nameSpace> --timeout 10000▼

The following table summarizes parameters used to configure a shared agent workspace. For more information, refer to Persistent storage.

Parameter Description

Parameter	Description
`storage.volumes.agentWorkspace.accessMode`	Define the workspace access mode. Possible values include `ReadWriteMany` and `ReadWriteOnce`. For shared workspaces use `ReadWriteMany`.
`storage.volumes.agentWorkspace.name`	The agent workspace name. Use the same name across all agents sharing the same workspace. If not specified, `flow-agent-workspace` is used. Specify the same name across all agents that share the workspace.
`storage.volumes.agentWorkspace.storage`	The amount of storage to allocate. For shared workspaces, allocate approximately 5 GiB per agent. Increase based on the agent’s requirements.
`storage.volumes.agentWorkspace.existingClaim`	Whether to use the existing claim for a previously deployed agent to share its workspace. Set to `true` to share the existing claim for `storage.volumes.agentWorkspace.name`.

storage.volumes.agentWorkspace.accessMode

Define the workspace access mode. Possible values include ReadWriteMany and ReadWriteOnce.

For shared workspaces use ReadWriteMany.

storage.volumes.agentWorkspace.name

The agent workspace name. Use the same name across all agents sharing the same workspace. If not specified, flow-agent-workspace is used.

Specify the same name across all agents that share the workspace.

storage.volumes.agentWorkspace.storage

The amount of storage to allocate.

For shared workspaces, allocate approximately 5 GiB per agent. Increase based on the agent’s requirements.

storage.volumes.agentWorkspace.existingClaim

Whether to use the existing claim for a previously deployed agent to share its workspace.

Set to true to share the existing claim for storage.volumes.agentWorkspace.name.

Configure an agent to use SSL/TLS certificates

Configuring SSL/TLS certificate in CloudBees CD/RO agent pods ensures secure communication between the agents and server.

Key considerations

Before configuring the certificates, note the following default settings:

Certificate location: Certificates are configured at the following locations:
- /opt/cbflow/conf/agent.crt
- /opt/cbflow/conf/agent.key
Configuration location: The SSL certificate configuration defined in values.yaml under the agent chart configuration.

Default value: SSL certificate configuration is disabled by default (enabled: false).

## Configure custom SSL/TLS certificates for agent HTTPS communication (port 7800).
## By default, agents auto-generate self-signed certificates during startup.
certificates:
enabled: false
existingSecret: ""
cert: ""
key: ""▼

You can configure the SSL/TLS certificates using any one of the following three methods.

Method 1: Use an existing Kubernetes secret

Create a Kubernetes secret containing the certificate and key, then mount it into the agent pods and configure Helm to use the secret.

Create a Kubernetes TLS secret using the following command:

kubectl create secret tls my-agent-certs \
--cert=path/to/agent.crt \
--key=path/to/agent.key \
--namespace=<namespace>▼

Configure Helm to use the existing secret using the following command:

# values.yaml
certificates:
  enabled: true
  existingSecret: my-agent-certs▼

Install or upgrade the agent using the following command:

helm install flow-agent cloudbees/cloudbees-flow-agent \
  -f values.yaml \
  --namespace=<namespace>▼

Method 2: Upload certificates during installation

Upload certificates at installation time without creating a Kubernetes secret as a prerequisite. This method keeps sensitive certificate data out of Helm charts and is recommended for production environments:

helm install flow-agent cloudbees/cloudbees-flow-agent \
  --set certificates.enabled=true \
  --set-file certificates.cert=path/to/agent.crt \
  --set-file certificates.key=path/to/agent.key \
  --namespace=<namespace>▼

Method 3: Add certificates in `values.yaml` file

Specify the certificate and keys directly in the values.yaml file.

This method is less secure and is not recommended for production environments.

# values.yaml
certificates:
  enabled: true
  cert: |
    -----BEGIN CERTIFICATE-----
    MIIDXTCCAkWgAwIBAgIJAKJ...
    -----END CERTIFICATE-----
  key: |
    -----BEGIN RSA PRIVATE KEY-----
    MIIEpAIBAAKCAQEA1q...
    -----END RSA PRIVATE KEY-----▼

Agent configuration

Configure internal and external agents

Install gateway agent pairs using the CloudBees CD/RO Helm chart

Install gateway agents using the CloudBees CD/RO agent Helm chart

Autoscaling gateway agents

Configuring HPA gateway agents

Autoscaling non-gateway agents

Configure agent resource name templates

Configure existing secrets for bound agents

Configure graceful termination period for agent pods

Configure agents to share a workspace

Configure an agent to use SSL/TLS certificates

Key considerations

Method 1: Use an existing Kubernetes secret

Method 2: Upload certificates during installation

Method 3: Add certificates in values.yaml file

Method 3: Add certificates in `values.yaml` file