Networking and Ingress

Network configuration and ingress management are critical for exposing CloudBees CD/RO services and controlling traffic flow in Kubernetes deployments. This page explains how to configure load balancing, manage ingress controllers, implement network policies, and transition to the Kubernetes Gateway API. These configurations enable you to secure communication between CloudBees CD/RO components and external clients while maintaining compliance with your organization’s networking standards.

Configure load balancing with ingress

For the default CloudBees CD/RO Helm charts, refer to Configure Helm charts. You can find these cloudbees-flow values referenced in the values.yaml under the Flow ingress configuration section.

An ingress controller typically does not eliminate the need for an external load balancer but adds a layer of routing and control behind the load balancer. However, you can configure the ingress controller so all service endpoints, such as web, server, and repository, may be exposed from the same domain name and load balancer endpoint.

CloudBees recommends configuring your ingress controller so all CloudBees CD/RO services can be exposed through a single load balancer.

By default, ingress is enabled in the CloudBees CD/RO chart. The following is a summary of the settings:

To run CloudBees CD/RO, you must use the ingress-nginx controller with the following required settings:

ingress-nginx.enabled=true
ingress.class=nginx

Parameter Default Value

Parameter	Default Value
ingress-nginx.controller.ingressClass	Default: `flow-ingress`
ingress-nginx.controller.publishService.enabled	Default: `true`
ingress-nginx.controller.scope.enabled	Default: `true`
ingress-nginx.enabled	Default: `true`
ingress-nginx.tcp.61613	CloudBees CD/RO server Default: `{{ .Release.Namespace }}/flow-server:61613`
ingress-nginx.tcp.8200	CloudBees CD/RO repository Default: `{{ .Release.Namespace }}/flow-repository:8200`
ingress-nginx.tcp.8443	CloudBees CD/RO web server Default: `{{ .Release.Namespace }}/flow-server:8443`
ingress-nginx.tcp.9201	CloudBees Analytics database Default: `{{ .Release.Namespace }}/flow-analytics:9201`

ingress-nginx.controller.ingressClass

Default: flow-ingress

ingress-nginx.controller.publishService.enabled

Default: true

ingress-nginx.controller.scope.enabled

Default: true

ingress-nginx.enabled

Default: true

ingress-nginx.tcp.61613

CloudBees CD/RO server

Default: {{ .Release.Namespace }}/flow-server:61613

ingress-nginx.tcp.8200

CloudBees CD/RO repository

Default: {{ .Release.Namespace }}/flow-repository:8200

ingress-nginx.tcp.8443

CloudBees CD/RO web server

Default: {{ .Release.Namespace }}/flow-server:8443

ingress-nginx.tcp.9201

CloudBees Analytics database

Default: {{ .Release.Namespace }}/flow-analytics:9201

Install custom ingress controllers with CloudBees CD/RO

To configure CloudBees CD/RO to use your custom or existing ingress controller:

Configure your ingress controller manifest to expose the following required TCP ports:

TCP port 8200 is required by flow-repository.
TCP ports 8443 and 61613 are required by flow-server.

(Optional) If you are using CloudBees Analytics, TCP port 9201 is required by flow-analytics.

If you need help configuring the ingress controller manifest, refer to Exposing TCP and UDP services in the NGINX Ingress Controller documentation. Ensure you follow the example for TCP/UDP proxy support.

Update your myvalues.yaml with your custom ingress controller values:
```
ingress:
  enabled: true
  host: <your-host>
  annotations: <your-annotations>
  class: <your-custom-ingress-class>▼
```
For a reference, refer to CloudBees CD/RO Helm chart ingress example.
In your myvalues.yaml, ensure ingress-nginx.enabled is configured as false for your installation.
```
ingress-nginx:
  enabled: false▼
```
If you do not have the ingress-nginx.enabled entry in your myvalues.yaml, by default, it is set to false in the cloudbees-flow values file.
If you have not already, apply your ingress controller manifest to your Kubernetes CloudBees CD/RO cluster.

To apply the custom ingress controller configuration, update your CloudBees CD/RO installation:

helm upgrade <server-releasename> cloudbees/cloudbees-flow \
-f <myvalues.yaml> \
--namespace <server-namespace> \
--timeout 10000s▼

Override network policies for CloudBees CD/RO on Kubernetes

Starting with CloudBees CD/RO release 2023.04, you can override network polices for CloudBees CD/RO on Kubernetes by adding custom network polices and ingress controllers. This capability enables you to customize network traffic management to meet your deployment requirements, such as using custom ingress controllers for internal communication between CloudBees CD/RO components or agents.

CloudBees strongly recommends validating all network policy changes in a development environment before releasing them into production. This helps ensure that the changes do not unintentionally disrupt communication between CloudBees CD/RO components, agents, or external services.

Before you begin

Before overriding the default CloudBees CD/RO network policies, ensure the following prerequisites are met:

You have one or more NetworkPolicy manifest(s) that defines network behavior for all CloudBees CD/RO components.

Example NetworkPolicy manifest

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  annotations:
    meta.helm.sh/release-name: cbflow
    meta.helm.sh/release-namespace: cbflow
  labels:
    app.kubernetes.io/managed-by: Helm
  name: web-policy
  namespace: cbflow
spec:
  ingress:
  - from:
    - ipBlock:
        cidr: 0.0.0.0/0
    - podSelector:
        matchLabels:
          app: flow-bound-agent
          release: cbflow
    - podSelector:
        matchLabels:
          app: flow-server
          release: cbflow
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: kube-system
      podSelector:
        matchLabels:
          app.kubernetes.io/name: ingress-nginx
    ports:
    - port: 2080
      protocol: TCP
    - port: 2443
      protocol: TCP
  podSelector:
    matchLabels:
      app: flow-web
      release: cbflow
  policyTypes:
  - Ingress▼

Your NetworkPolicy manifest(s) are present in the cluster or accessible from the environment where you manage the cluster. To apply manifest(s) to the cluster, run the following command:
```
kubectl apply -f <your-NetworkPolicy-manifest.yaml> \
      --namespace <development-server-namespace>▼
```
If you have multiple manifest files, specify each additional file using the -f option in the command.

If your myvalues.yaml is based on the cloudbees-flow values file version 2.23.1 or earlier, you must update the Network isolation configuration section with the latest configuration settings:

Navigate to Sonatype Nexus and download the latest cloudbees-flow release.
Extract the package, and open the cloudbees-flow/values.yaml with your existing myvalues.yaml.
Update your myvalues.yaml file with the required configuration changes before proceeding with the deployment or upgrade.

When updating your myvalues.yaml with the latest Network isolation configuration section, ensure:

The existing custom networkIsolation settings are preserved and not overwritten.
The updated values file has a valid YAML and all indentation is correct. Ensure that this step is completed before upgrading; otherwise, the upgrade may fail or produce unexpected results.

Override CloudBees CD/RO Kubernetes network policies

After you’ve met the requirements in [override-network-policies-prereqs], perform the following in the myvalues.yaml:

Navigate to the Network isolation configuration section.

To disable the default CloudBees CD/RO network, configure networkIsolation.networkPolicies.enable: false.

After configuring networkIsolation.networkPolicies.enable: false, CloudBees CD/RO will not create its default network policies, and depend completely on your custom policies. Failure to correctly configure your custom network policies may cause unexpected behavior in your deployment.

In the additionalIngressSelector for each CloudBees CD/RO component, provide the network policy configurations for your ingress.

Example custom cbflow-web ingress configuration

  web:
    additionalIngressSelector:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: kube-system
      podSelector:
        matchLabels:
          app.kubernetes.io/name: ingress-nginx▼

Once you’ve updated your myvalues.yaml, update your flow-server deployment:

helm upgrade <development-server-releasename> cloudbees/cloudbees-flow \
      -f <myvalues.yaml> \
      --namespace <development-server-namespace> \
      --timeout 10000s▼

You can now test your custom network policies in your development environment. Once tested, to update the production servers, refer to Install CloudBees CD/RO within Kubernetes.

Configure ingress-nginx default namespace-specific

CloudBees does not suggest installing multiple instances of CloudBees CD/RO in different namespaces within the same Kubernetes cluster, especially in production environments. This can create multiple naming conflicts among components.

If you install multiple CloudBees CD/RO instances in different namespaces on the same Kubernetes cluster, the installation can fail due to ingress-nginx naming conflicts. By default, each instance uses the same ingress-nginx class name defined in the flow-server values file. Because the ingress-nginx.controller.ingressClassResource.name value must be unique within a Kubernetes cluster, you must configure a unique name for each CloudBees CD/RO instance in your non-production environment.

To resolve this, use the --set directive in the helm install command to provide a unique name for ingress-nginx.controller.ingressClassResource.name in each CloudBees CD/RO instance. This ensures the ingress class resources do not conflict when multiple CloudBees CD/RO instances are deployed within the the same cluster.

When installing or upgrading a CloudBees CD/RO instance in a non-production environment, refer to the following example and naming conventions:

RELEASE="<YOUR-CD/RO-RELEASE>"
NAMESPACE="<YOUR-CD/RO-NAMESPACE>"

helm install $RELEASE cloudbees/cloudbees-flow \
--values=myvalues.yaml \
--namespace $NAMESPACE \
--set ingress-nginx.controller.ingressClassResource.name=$RELEASE.$NAMESPACE▼

Resolve an ingress class name conflict

If you have an ingress class name conflict during your upgrade:

Update the default ingress class name for the ingress-nginx controller with the additional values in your myvalues.yaml.

ingress:
  enabled: true
  host: <your-host.example.com>
  class: <ingress-class-name>

ingress-nginx:
  enabled: true
  controller:
    ingressClassResource:
      name: <ingress-class-name>▼

Patch the existing ingress by running the following command:

kubectl patch ingress/flow-ingress -p '{"spec": {"ingressClassName":"<ingress-class-name>" }}' -n <namespace>▼

If the naming conflict persists, verify that the value of ingress-nginx.controller.ingressClassResource.name matches the value specified for ingress.class, and both values are configured correctly. If you updated myvalues.yaml to correct the ingress class names, rerun the previous installation or upgrade command to apply the changes and resolve the conflict.

Resolve DNS/SSL issues after resolving ingress class name conflicts

If you have a DNS/SSL issue after resolving the ingress class name conflict, restart your ingress-nginx pod to mount the local certificates:

Identify the ingress-nginx controller deployment using the following command:

kubectl get deployments -n <namespace> | grep ingress-nginx▼

This returns the ingress-nginx deployment, such as:

<your-deployment>-ingress-nginx-controller   1/1     1            1           56m▼

Restart the ingress-nginx controller deployment to apply the updated configuration:

kubectl rollout restart deployment <your-deployment>-ingress-nginx-controller -n <namespace>▼

Verify that the deployment restarts successfully and that the controller pods return to a healthy state.
The upgrade is complete.

Configure Kubernetes Gateway API

The upstream ingress-NGINX project reached end-of-life (EOL) in March 2026 and no longer receives updates, security patches, or technical support. Although existing deployments might continue to function, CloudBees does not provide support for issues related to ingress-NGIN. CloudBees recommends migrating to a supported ingress controller to ensure continued security, maintenance, and vendor support.

What this means:

Ingress-NGINX deployments continue to operate, but are no longer maintained or patched upstream.
CloudBees does not provide technical support for ingress-NGINX configurations.
The bundled ingress-NGINX subchart will be removed from the cloudbees-flow chart in future releases.

CloudBees strongly recommends that you begin evaluating and configuring the Kubernetes Gateway API, along with the required controller-specific policies.

Starting with CloudBees CD/RO release 2026.03.0, you must manage your ingress or Gateway API controller independently. If currently using the ingress-NGINX controller bundled with cloudbees-flow, verify whether ingress-nginx.enabled: true is configured in your Helm values. In CloudBees CD/RO version 2026.03.0, ingress-NGINX continues to be the default option while the Gateway API configuration is optional.

In CloudBees CD/RO version 2026.03.0, ingress-NGINX remains the default ingress option for existing deployments, while Gateway API support is available as an optional configuration. However, because ingress-NGINX has reached end-of-life, CloudBees recommends planning a migration to a supported ingress or Gateway API controller to ensure continued security updates, maintenance, and support.

When configuring the Kubernetes Gateway API, CloudBees recommends validating all Gateway API components in a non-production environment before enabling them in production. Configuration changes involving gateways, listeners, routes, or policies can directly affect application availability and user experience.

Before deploying the configuration to production, verify critical workflows such as file uploads, long-running operations, Commander UI sessions, and ectool connectivity. Testing these workflows helps ensure that traffic routing, session handling, timeouts, and network policies are configured correctly and operate as expected in your environment. If you have questions or need assistance, please contact CloudBees Support.

Roles and personas

Kubernetes Gateway API uses a role-based ownership model. CloudBees being an application developer, the following are our responsibilities:

Owner Responsibility

Owner	Responsibility
CloudBees CD/RO `cloudbees-flow` chart	The `cloudbees-flow` chart creates only routes (HTTPRoute, TLSRoute, and TCPRoute resources). It does not create a Gateway, GatewayClass, or any controller-specific policies. You must create and manage these resources.
User (Cluster operator)	GatewayClass, Gateway resource, TLS certificates, controller-specific policies.

CloudBees CD/RO cloudbees-flow chart

The cloudbees-flow chart creates only routes (HTTPRoute, TLSRoute, and TCPRoute resources). It does not create a Gateway, GatewayClass, or any controller-specific policies. You must create and manage these resources.

User (Cluster operator)

GatewayClass, Gateway resource, TLS certificates, controller-specific policies.

Prerequisites

Following are the prerequisites for configuring Kubernetes Gateway API:

Kubernetes 1.31 and later versions for the chart’s default TLSRoute (v1) (the standard-channel TLSRoute CRD uses Common Expression Language (CEL) validation rules introduced in Kubernetes 1.31 version). For more information, refer to the Gateway API v1.5.0 release notes. Kubernetes 1.27 and later is sufficient for HTTPRoute and TCPRoute.
Install Gateway API CustomResourceDefinitions (CRDs). Before proceeding, refer to the Official Gateway API Installation Guide to confirm the latest version. The following examples use version v1.5.0. Choose one channel:
- Standard channel (recommended; supports HTTPRoute and the chart’s default TLSRoute):
  kubectl apply --server-side -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.5.0/standard-install.yaml
  ▼
- Experimental channel (required only if you switch any port to TCPRoute, which is still v1alpha2):
  kubectl apply --server-side -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.5.0/experimental-install.yaml
  ▼
A Gateway API controller installed and a Gateway resource created. For more information, refer to Supported and tested gateway controllers.
A TLS Secret accessible to the Gateway’s HTTPS listener. The following options are supported:
- Same namespace as CloudBees CD/RO: Reference the secret directly in certificateRefs. No additional configuration is required and you can reuse the existing ingress TLS secret if it already resides in the same namespace.
- Different namespace: Create a ReferenceGrant in the secret’s namespace to allow the Gateway to reference it. For more information, refer to Cross-namespace reference guide.

Key components

CloudBees CD/RO creates the following components:

HTTPRoute: Routes HTTP/HTTPS traffic to flow-web. Supports SDA mode (paths /flow, /commander, /auth, /rest) and non-SDA mode (catch-all /).

TLSRoutes (default, per-port): Routes TCP traffic to backend pods that terminate TLS themselves (such as flow-server and flow-repository). Routing decisions are made on the Server Name Indication (SNI) hostname in the TLS ClientHello, so multiple CloudBees CD/RO releases can share a single LoadBalancer IP and port. The Gateway listener does NOT decrypt — TLS continues to terminate at the backend pod, and existing certificate management is preserved.

Verify that your controller supports TLSRoute: Not every Gateway API controller has implemented TLSRoute. Review your controller’s release notes and conformance report. For more information, refer Supported and tested gateway controllers. If your controller does not support TLSRoute, switch to TCPRoute by setting tlsRoute.<svc>.enabled: false and tcpRoute.<svc>.enabled: true for the affected ports.

Clients MUST send SNI: TLSRoute matches traffic based on the Server Name Indication (SNI) extension in the TLS ClientHello. Connections that do not include SNI, such as curl example:https://<gateway-ip:8443/, do not match any TLSRoute rule and are dropped. However, ectool, web browsers, and most HTTPS clients automatically send SNI when connecting to a hostname. Update any legacy scripts that connect using a raw IP address to use the hostname instead.

Route name Port Backend Default sectionName

Route name	Port	Backend	Default sectionName
`flow-tlsroute-secure-server`	8443	`flow-server`	`flow-secure-server`
`flow-tlsroute-stomp`	61613	`flow-server`	`flow-stomp`
`flow-tlsroute-repository`	8200	`flow-repository`	`flow-repository`

flow-tlsroute-secure-server

8443

flow-server

flow-secure-server

flow-tlsroute-stomp

61613

flow-server

flow-stomp

flow-tlsroute-repository

8200

flow-repository

Additional TLSRoutes: Optional TLSRoutes for components such as DOIS, Analytics, or external agents can be configured using gatewayApi.tlsRoute.additional[], mirroring the tcpRoute.additional[] structure.

TCPRoutes (alternative, per-port): Use TCPRoute for ports where the backend does NOT terminate TLS (raw TCP traffic), where clients cannot send SNI, or where the Gateway listener uses protocol: TCP and TLS Passthrough is unavailable. Built-in TCPRoutes are not enabled by default; opt in per service by setting gatewayApi.tcpRoute.<svc>.enabled: true and gatewayApi.tlsRoute.<svc>.enabled: false.

Route name Port Backend Default sectionName

Route name	Port	Backend	Default sectionName
`flow-tcproute-secure-server`	8443	`flow-server`	`flow-secure-server`
`flow-tcproute-stomp`	61613	`flow-server`	`flow-stomp`
`flow-tcproute-repository`	8200	`flow-repository`	`flow-repository`

flow-tcproute-secure-server

8443

flow-server

flow-secure-server

flow-tcproute-stomp

61613

flow-server

flow-stomp

flow-tcproute-repository

8200

flow-repository

Additional TCPRoutes: Configures optional components using gatewayApi.tcpRoute.additional[].

Component Port Service

Component	Port	Service
External agent	7800	`gateway-external-agent-flow-agents`
DOIS Elasticsearch	9200	`flow-devopsinsight`
Analytics	9201	`flow-analytics`
Insecure server	8000	`flow-server`

External agent

7800

gateway-external-agent-flow-agents

DOIS Elasticsearch

9200

flow-devopsinsight

Analytics

9201

flow-analytics

Insecure server

8000

flow-server

Per-port mutex: For any given service, you cannot enable both tcpRoute.<svc>.enabled and tlsRoute.<svc>.enabled. The chart fails helm template with an actionable error because a Gateway listener accepts only one route kind per port. Mixed configurations are supported — for example, server and stomp on TLSRoute while repository runs on TCPRoute.

Points to remember

Following are some points to note when configuring Gateway API:

TLSRoute API version: gateway.networking.k8s.io/v1 since Gateway API v1.5.0 (Standard channel, GA per GEP-2643). To target older Gateway API CRDs that still expose TLSRoute as v1alpha2, override gatewayApi.tlsRoute.apiVersion in your Helm values.
TCPRoute API version: Currently v1alpha2 (experimental channel). When TCPRoute reaches a stable version, CloudBees will update the default value in the chart. If needed, you can update gatewayApi.tcpRoute.apiVersion in your Helm values at any time to match the latest TCPRoute API version available in your cluster.
Session persistence standardization: GEP-1619 introduces a standard sessionPersistence field on HTTPRoute. When implemented correctly, cookie-based session affinity will become portable without requiring controller-specific policies.

Enable Gateway API

The following Helm values are required to enable Gateway API:

Required Helm values

gatewayApi:
  enabled: true
  gatewayName: "<your-gateway-name>"     # REQUIRED — name of the Gateway resource
  host: "cdro.example.com"               # REQUIRED — hostname for routes▼

Full value reference

To see the complete Gateway API default values from the chart, use the following command:

helm repo update
helm show values cloudbees/cloudbees-flow | sed -n '/### Gateway API configuration section/,/### Miscellaneous configuration section/p' | grep -v "Miscellaneous"▼

The following table provides the default Gateway API values and its description:

Value Default Description

Value	Default	Description
`gatewayApi.enabled`	false	Enable route creation
`gatewayApi.host`	""	Hostname for routes (required)
`gatewayApi.gatewayName`	""	Gateway resource name (required)
`gatewayApi.gatewayNamespace`	release namespace	Gateway namespace
`gatewayApi.httpRoute.enabled`	true	Create HTTPRoute
`gatewayApi.httpRoute.hostnames`	`[gatewayApi.host]`	Override hostnames
`gatewayApi.tlsRoute.apiVersion`	gateway.networking.k8s.io/v1	Standard channel since Gateway API v1.5.0
`gatewayApi.tlsRoute.server.enabled`	true	Default for port 8443; mutually exclusive with `tcpRoute.server.enabled`.
`gatewayApi.tlsRoute.server.sectionName`	flow-secure-server	Must match Gateway TLS listener name.
`gatewayApi.tlsRoute.server.hostnames`	`[]` (falls back to `gatewayApi.host`)	SNI hostnames matched by this TLSRoute.
`gatewayApi.tlsRoute.stomp.enabled`	true	Default for port 61613; mutually exclusive with `tcpRoute.stomp.enabled`.
`gatewayApi.tlsRoute.stomp.sectionName`	flow-stomp	Must match Gateway TLS listener name.
`gatewayApi.tlsRoute.repository.enabled`	true	Default for port 8200; mutually exclusive with `tcpRoute.repository.enabled`.
`gatewayApi.tlsRoute.repository.sectionName`	flow-repository	Must match Gateway TLS listener name.
`gatewayApi.tlsRoute.additional`	`[]`	Additional TLSRoutes.
`gatewayApi.tcpRoute.apiVersion`	gateway.networking.k8s.io/v1alpha2	Update when TCPRoute graduates
`gatewayApi.tcpRoute.server.enabled`	false	Set to `true` (and `tlsRoute.server.enabled: false`) to use TCPRoute on port 8443.
`gatewayApi.tcpRoute.server.sectionName`	flow-secure-server	Must match Gateway listener name
`gatewayApi.tcpRoute.stomp.enabled`	false	Set to `true` (and `tlsRoute.stomp.enabled: false`) to use TCPRoute on port 61613.
`gatewayApi.tcpRoute.stomp.sectionName`	flow-stomp	Must match Gateway listener name
`gatewayApi.tcpRoute.repository.enabled`	false	Set to `true` (and `tlsRoute.repository.enabled: false`) to use TCPRoute on port 8200.
`gatewayApi.tcpRoute.repository.sectionName`	flow-repository	Must match Gateway listener name
`gatewayApi.tcpRoute.additional`	`[]`	Additional TCP routes

gatewayApi.enabled

false

Enable route creation

gatewayApi.host

Hostname for routes (required)

gatewayApi.gatewayName

Gateway resource name (required)

gatewayApi.gatewayNamespace

release namespace

Gateway namespace

gatewayApi.httpRoute.enabled

true

Create HTTPRoute

gatewayApi.httpRoute.hostnames

[gatewayApi.host]

Override hostnames

gatewayApi.tlsRoute.apiVersion

gateway.networking.k8s.io/v1

Standard channel since Gateway API v1.5.0

gatewayApi.tlsRoute.server.enabled

true

Default for port 8443; mutually exclusive with tcpRoute.server.enabled.

gatewayApi.tlsRoute.server.sectionName

flow-secure-server

Must match Gateway TLS listener name.

gatewayApi.tlsRoute.server.hostnames

[] (falls back to gatewayApi.host)

SNI hostnames matched by this TLSRoute.

gatewayApi.tlsRoute.stomp.enabled

true

Default for port 61613; mutually exclusive with tcpRoute.stomp.enabled.

gatewayApi.tlsRoute.stomp.sectionName

flow-stomp

Must match Gateway TLS listener name.

gatewayApi.tlsRoute.repository.enabled

true

Default for port 8200; mutually exclusive with tcpRoute.repository.enabled.

gatewayApi.tlsRoute.repository.sectionName

flow-repository

Must match Gateway TLS listener name.

gatewayApi.tlsRoute.additional

[]

Additional TLSRoutes.

gatewayApi.tcpRoute.apiVersion

gateway.networking.k8s.io/v1alpha2

Update when TCPRoute graduates

gatewayApi.tcpRoute.server.enabled

false

Set to true (and tlsRoute.server.enabled: false) to use TCPRoute on port 8443.

gatewayApi.tcpRoute.server.sectionName

flow-secure-server

Must match Gateway listener name

gatewayApi.tcpRoute.stomp.enabled

false

Set to true (and tlsRoute.stomp.enabled: false) to use TCPRoute on port 61613.

gatewayApi.tcpRoute.stomp.sectionName

flow-stomp

Must match Gateway listener name

gatewayApi.tcpRoute.repository.enabled

false

Set to true (and tlsRoute.repository.enabled: false) to use TCPRoute on port 8200.

gatewayApi.tcpRoute.repository.sectionName

flow-repository

Must match Gateway listener name

gatewayApi.tcpRoute.additional

[]

Additional TCP routes

Switch a port to TCPRoute

The chart enables TLSRoute by default for all three built-in services (server, stomp, and repository). To use TCPRoute for a service port instead, set the following options:

tlsRoute.<service>.enabled: false
tcpRoute.<service>.enabled: true▼

The following example uses default TLSRoute for all three built-in services. No tcpRoute or tlsRoute overrides are needed:

gatewayApi:
  enabled: true
  gatewayName: "flow-gateway"
  host: "cdro.example.com"▼

The following example shows a mixed configuration with server and stomp on the default TLSRoute and repository switched to TCPRoute:

gatewayApi:
  enabled: true
  gatewayName: "flow-gateway"
  host: "cdro.example.com"
  tlsRoute:
    repository:
      enabled: false   # disable TLSRoute for :8200 only
    # server and stomp default to true → stay on TLSRoute
  tcpRoute:
    repository:
      enabled: true    # enable TCPRoute for :8200 only
    # server and stomp default to false → no TCPRoute▼

The following example switches all three built-in services to TCPRoute (for example, when your controller does not support TLSRoute):

gatewayApi:
  enabled: true
  gatewayName: "flow-gateway"
  host: "cdro.example.com"
  tlsRoute:
    server:
      enabled: false
    stomp:
      enabled: false
    repository:
      enabled: false
  tcpRoute:
    server:
      enabled: true
    stomp:
      enabled: true
    repository:
      enabled: true▼

Gateway resources to create

Your Gateway must have listeners that match the sectionNames expected by the cloudbees-flow chart. Each listener name must exactly match the corresponding sectionName configured in the Helm values. The chart’s default route kind is TLSRoute, so the listeners on ports 8443, 61613, and 8200 use protocol: TLS with tls.mode: Passthrough. The TLS listeners do not include certificateRefs because the gateway is not terminating TLS — it matches on SNI and forwards the encrypted bytes to the backend pod, which terminates TLS itself.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: <your-gateway-name>
  namespace: <cdro-namespace>
spec:
  gatewayClassName: <your-controller-class>   # for example, istio, nginx, kgateway
  listeners:
    - name: http
      hostname: "cdro.example.com"
      port: 80
      protocol: HTTP
      allowedRoutes:
        namespaces:
          from: Same
    - name: https
      hostname: "cdro.example.com"
      port: 443
      protocol: HTTPS
      tls:
        mode: Terminate
        certificateRefs:
          - name: <tls-secret>
            kind: Secret
      allowedRoutes:
        namespaces:
          from: Same
    - name: flow-secure-server    # matches gatewayApi.tlsRoute.server.sectionName
      port: 8443
      protocol: TLS
      tls:
        mode: Passthrough
      allowedRoutes:
        namespaces:
          from: Same
        kinds:
          - kind: TLSRoute
    - name: flow-stomp            # matches gatewayApi.tlsRoute.stomp.sectionName
      port: 61613
      protocol: TLS
      tls:
        mode: Passthrough
      allowedRoutes:
        namespaces:
          from: Same
        kinds:
          - kind: TLSRoute
    - name: flow-repository       # matches gatewayApi.tlsRoute.repository.sectionName
      port: 8200
      protocol: TLS
      tls:
        mode: Passthrough
      allowedRoutes:
        namespaces:
          from: Same
        kinds:
          - kind: TLSRoute▼

If the Gateway is deployed in a different namespace than the CloudBees CD/RO deployment, the cluster operator must configure each Gateway listener’s allowedRoutes.namespaces setting to allow route attachment from the CloudBees CD/RO namespace.

For the simplest configuration, set from: All and for more restrictive access, set from: Selector and provide a label selector that matches the CloudBees CD/RO namespace.

For more information, refer to the Gateway API Multiple Namespaces guide.

TCPRoute listener variant

If you switch to TCP route, replace the corresponding TLS Passthrough listener with the TCP variant below. Per-port mixed listeners are supported — for example, flow-secure-server and flow-stomp can stay TLS while flow-repository is changed to TCP.

    - name: flow-repository       # matches gatewayApi.tcpRoute.repository.sectionName
      port: 8200
      protocol: TCP
      allowedRoutes:
        namespaces:
          from: Same
        kinds:
          - kind: TCPRoute▼

You can keep TCP and TLS listeners side by side on different ports if you need both kinds simultaneously. Some controllers (for example, Istio) also allow same-port co-existence by merging filter chains, but this behavior is controller-specific and not portable. Verify on your controller before relying on it.

Ingress-NGINX parity requirements

CloudBees CD/RO relies on the following Ingress-NGINX behaviors. Your Gateway API controller and policies must provide equivalent functionality, otherwise a failure might occur. The default behavior varies by controller. For more information, refer to Supported and tested gateway controllers.

Ingress-NGINX annotation Behavior Error Required action

Ingress-NGINX annotation	Behavior	Error	Required action
`proxy-body-size: 10G`	Allow uploads up to 10GB	Depending on the controller’s default limits, large uploads may trigger an HTTP 413 error.	Increase the upload limit greater than 10 GB using your controller’s policy mechanism.
`proxy-read-timeout: 4000`	Allow blocking API calls up to 4000 seconds	If the controller’s default timeout is lower than the operation duration, it triggers an HTTP 504 error.	Set request timeout greater than 4000 seconds using your controller’s policy mechanism.
`affinity: cookie`	Sticky sessions for Commander UI	Commander UI sign out on navigation	Configure session affinity using your controller’s supported mechanism.
`proxy-stream-timeout: 4000`	TCP idle connections stay open.	The controller’s 10-minute default was sufficient during validation. This behavior might vary.	Validate in your environment and configure if idle TCP connections drop unexpectedly.
`secure-backends: false`	Tells Ingress-NGINX to send plain HTTP (not HTTPS) to the backend server; TLS already terminated at the ingress.	Default behavior in Gateway API — HTTPRoute backendRef points to the backend’s HTTP port.	No action required.

proxy-body-size: 10G

Allow uploads up to 10GB

Depending on the controller’s default limits, large uploads may trigger an HTTP 413 error.

Increase the upload limit greater than 10 GB using your controller’s policy mechanism.

proxy-read-timeout: 4000

Allow blocking API calls up to 4000 seconds

If the controller’s default timeout is lower than the operation duration, it triggers an HTTP 504 error.

Set request timeout greater than 4000 seconds using your controller’s policy mechanism.

affinity: cookie

Sticky sessions for Commander UI

Commander UI sign out on navigation

Configure session affinity using your controller’s supported mechanism.

proxy-stream-timeout: 4000

TCP idle connections stay open.

The controller’s 10-minute default was sufficient during validation. This behavior might vary.

Validate in your environment and configure if idle TCP connections drop unexpectedly.

secure-backends: false

Tells Ingress-NGINX to send plain HTTP (not HTTPS) to the backend server; TLS already terminated at the ingress.

Default behavior in Gateway API — HTTPRoute backendRef points to the backend’s HTTP port.

No action required.

Supported and tested gateway controllers

The following controller configurations reflect what was working at the time of validation. Some features referenced here are still in the experimental channel of the Gateway API spec and may change in future controller releases.

CloudBees CD/RO provides these examples for reference only and does not guarantee that they will work across all versions. You are responsible for configuring and managing the Gateway, GatewayClass, TLS, policies, and all controller-specific settings, including deployment, maintenance, and validation.

CloudBees CD/RO is responsible only for the HTTPRoute, TCPRoute, and TLSRoute resources created by the cloudbees-flow chart.

Controller	Upload > 1MB	Timeout > 60s	Session Affinity	TLSRoute	Policies Required	Production Ready
[Istio] (>= 1.30)	No policy needed	No policy needed.	DestinationRule (cookie)	(Istio >= 1.30)	1
[NGF]	ClientSettings policy	Snippets policy	UpstreamSettings policy (ip_hash)	Verify with NGF release notes	3
[Traefik]	No policy needed	No policy needed	Not supported		0	No session affinity
[kgateway]	TrafficPolicy	TrafficPolicy	Broken (cookie Path bug)	Verify with kgateway release notes	2	No session affinity

Controller

Upload > 1MB

Timeout > 60s

Session Affinity

TCPRoute

TLSRoute

Policies Required

Production Ready

[Istio] (>= 1.30)

No policy needed

No policy needed.

DestinationRule (cookie)

(Istio >= 1.30)

[NGF]

ClientSettings policy

Snippets policy

UpstreamSettings policy (ip_hash)

Verify with NGF release notes

[Traefik]

No policy needed

Not supported

No session affinity

[kgateway]

TrafficPolicy

Broken (cookie Path bug)

Verify with kgateway release notes

No session affinity

TLSRoute support is a fast-moving area — controllers are still rolling out conformance for the v1 standard-channel resource. Before opting in, verify your controller version’s TLSRoute support against the controller’s own release notes or Gateway API conformance report.

Istio: Fully supported with one policy required

Istio is Envoy-based. During CloudBees' validation, Envoy did not enforce a default upload body size limit, and Istio had request timeouts disabled by default. Based on this behavior, only session affinity required additional policy configuration. Verify that these defaults apply to your specific Istio version. For more information, refer to Istio documentation and Istio Gateway API guide.

Version requirement for TLSRoute: Istio 1.30 or later is required if you opt into TLSRoute. Earlier Istio versions paired with Gateway API v1.5+ standard CRDs are broken (Istio issue #59387) — if Istio fails to start because it looks for TLSRoute at v1alpha2, which v1.5+ removed from the standard channel. If you are on Istio 1.29 or earlier, stay on TCPRoute until you can upgrade.

Perform the following additional steps for CloudBees CD/RO deployment:

Enable alpha Gateway API features (required for TCPRoute) using the following command:

helm upgrade istiod istio/istiod -n istio-system \
  --set pilot.env.PILOT_ENABLE_ALPHA_GATEWAY_API=true \
  --reuse-values▼

Configure the required policy for session affinity, using the following command:

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: flow-web-sticky
  namespace: <cdro-namespace>
spec:
  host: flow-web.<cdro-namespace>.svc.cluster.local
  trafficPolicy:
    loadBalancer:
      consistentHash:
        httpCookie:
          name: ROUTE
          path: /
          ttl: 3600s▼

The following table gives CloudBees CD/RO requirements for configuring Istio:

CloudBees CD/RO requirement Status Notes

CloudBees CD/RO requirement	Status	Notes
Upload >1MB	No policy needed	Envoy has no default body size limit.
Timeout >60s	No policy needed	Istio disables request timeout by default
Session affinity	DestinationRule	Cookie-based, functionally equivalent to Ingress-NGINX
TCP ports (8443, 61613, 8200)		True TCP passthrough; requires the alpha Gateway API flag enabled above.
TLSRoute (per-port opt-in)	(Istio >= 1.30)	Listener `protocol: TLS, tls.mode: Passthrough`. Earlier Istio versions are broken with Gateway API v1.5+ standard CRDs (issue #59387).

Upload >1MB

No policy needed

Envoy has no default body size limit.

Timeout >60s

No policy needed

Istio disables request timeout by default

Session affinity

DestinationRule

Cookie-based, functionally equivalent to Ingress-NGINX

TCP ports (8443, 61613, 8200)

True TCP passthrough; requires the alpha Gateway API flag enabled above.

TLSRoute (per-port opt-in)

(Istio >= 1.30)

Listener protocol: TLS, tls.mode: Passthrough. Earlier Istio versions are broken with Gateway API v1.5+ standard CRDs (issue #59387).

NGINX Gateway Fabric (NGF): Fully supported with limitation in OSS version

For NGF Helm installation, refer to Installing NGF.

Perform the following additional steps for CloudBees CD/RO deployment:

Enable experimental features and SnippetsPolicy, using the following command:

helm upgrade ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric \
  --namespace nginx-gateway \
  --set nginxGateway.gwAPIExperimentalFeatures.enable=true \
  --set nginxGateway.snippets.enable=true \
  --reuse-values▼

Apply the three required policies, using the following command:

# Policy 1: Upload size (replaces proxy-body-size: 10G)
apiVersion: gateway.nginx.org/v1alpha1
kind: ClientSettingsPolicy
metadata:
  name: flow-client-settings
  namespace: <cdro-namespace>
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: HTTPRoute
    name: flow-httproute
  body:
    maxSize: "10g"
---
# Policy 2: Request timeout (replaces proxy-read-timeout: 4000)
# Must target Gateway (not HTTPRoute) — CRD constraint
apiVersion: gateway.nginx.org/v1alpha1
kind: SnippetsPolicy
metadata:
  name: flow-proxy-timeouts
  namespace: <cdro-namespace>
spec:
  targetRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: <your-gateway-name>
  snippets:
    - context: http.server.location
      value: |
        proxy_read_timeout 4000s;
        proxy_send_timeout 4000s;
---
# Policy 3: Session affinity (replaces affinity: cookie)
# Note: ip_hash — not cookie-based. All requests from same client IP → same web server pod.
# Acceptable for enterprise deployments; may cause uneven load if clients share NAT IP.
apiVersion: gateway.nginx.org/v1alpha1
kind: UpstreamSettingsPolicy
metadata:
  name: flow-upstream-settings
  namespace: <cdro-namespace>
spec:
  targetRefs:
    - group: ""
      kind: Service
      name: flow-web
  loadBalancingMethod: "ip_hash"▼

The following table gives CloudBees CD/RO requirements for configuring NGF:

CloudBees CD/RO requirement Status Notes

CloudBees CD/RO requirement	Status	Notes
Upload >1MB	ClientSettingsPolicy	`body.maxSize: "10g"`
Timeout >60s	SnippetsPolicy	Raw NGINX config injection; SnippetsPolicy must be explicitly enabled
Session affinity	UpstreamSettingsPolicy	`ip_hash` - IP-based, not cookie-based
TCP ports (8443, 61613, 8200)		True TCP passthrough

Upload >1MB

ClientSettingsPolicy

body.maxSize: "10g"

Timeout >60s

SnippetsPolicy

Raw NGINX config injection; SnippetsPolicy must be explicitly enabled

Session affinity

UpstreamSettingsPolicy

ip_hash - IP-based, not cookie-based

TCP ports (8443, 61613, 8200)

True TCP passthrough

Allows raw NGINX config injection. Disabled by default. Evaluate whether this is acceptable for your security posture. NGINX Plus alternative: native timeout policies without raw injection.

Following are the known limitations and considerations for Open Source (OSS) Controllers:

Limitation Impact Alternative

Limitation	Impact	Alternative
Cookie-based sticky sessions require NGINX Plus	OSS uses `ip_hash` for all requests from the same client IP to go to the same pod. Clients behind shared NAT may cluster on one pod causing uneven load.	NGINX Gateway Fabric Plus (native `sticky cookie`), Istio (`DestinationRule`), or accept `ip_hash trade-offs` for enterprise deployments.
`SnippetsPolicy` allows raw NGINX config injection	Security risk in multi-tenant clusters; must be explicitly enabled.	NGINX Gateway Fabric Plus has native timeout policies without raw injection.
Policies are NGF-specific CRDs	`ClientSettingsPolicy`, `SnippetsPolicy`, `UpstreamSettingsPolicy` are not portable to other controllers	Each controller has its own policy mechanism.

Cookie-based sticky sessions require NGINX Plus

OSS uses ip_hash for all requests from the same client IP to go to the same pod. Clients behind shared NAT may cluster on one pod causing uneven load.

NGINX Gateway Fabric Plus (native sticky cookie), Istio (DestinationRule), or accept ip_hash trade-offs for enterprise deployments.

SnippetsPolicy allows raw NGINX config injection

Security risk in multi-tenant clusters; must be explicitly enabled.

NGINX Gateway Fabric Plus has native timeout policies without raw injection.

Policies are NGF-specific CRDs

ClientSettingsPolicy, SnippetsPolicy, UpstreamSettingsPolicy are not portable to other controllers

Each controller has its own policy mechanism.

Traefik v3: Limited — session affinity not supported

For more information about installing Traefik, refer to Traefik documentation.

Perform the following additional steps for CloudBees CD/RO deployment:

Configure entryPoints: Define Traefik entryPoints that match the Gateway listener names and ports.
Update Helm values: Customize the Traefik Helm values to include the required entryPoints before creating the Gateway.
Choose the route kind for each TCP port:
- TLSRoute (default in the chart, supported on the Standard channel since Traefik 3.7) is recommended when the backend terminates TLS itself and clients can send SNI. The Gateway listener must use protocol: TLS with tls.mode: Passthrough for ports configured for TLSRoute.
- TCPRoute is the alternative for raw TCP backends or clients that cannot send SNI. TCPRoute on Traefik requires the experimental channel — set the Kubernetes Gateway provider’s experimentalChannel: true flag.

For more information, refer to the Traefik Kubernetes Gateway provider documentation.

kgateway v2.3: Limited session affinity broken

For more information about installing kgateway v2.3, refer to kgateway documentation.

Perform the following additional steps for CloudBees CD/RO deployment:

Enable experimental features (required for TCPRoute and GEP-1619 session persistence), using the following command:

helm upgrade -i -n kgateway-system kgateway \
  oci://cr.kgateway.dev/kgateway-dev/charts/kgateway \
  --version v2.3.0-main \
  --set controller.extraEnv.KGW_ENABLE_GATEWAY_API_EXPERIMENTAL_FEATURES=true▼

Apply two required policies, using the following command:

# Policy 1: Upload size (max ~4GB due to Envoy int32 limit)
apiVersion: gateway.kgateway.dev/v1alpha1
kind: TrafficPolicy
metadata:
  name: flow-buffer-limits
  namespace: <cdro-namespace>
spec:
  targetRefs:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
      name: flow-httproute
  buffer:
    maxRequestSize: "4095Mi"
---
# Policy 2: Request timeout (Envoy default is 15s — more aggressive than NGF's 60s)
apiVersion: gateway.kgateway.dev/v1alpha1
kind: TrafficPolicy
metadata:
  name: flow-proxy-timeout
  namespace: <cdro-namespace>
spec:
  targetRefs:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
      name: flow-httproute
  timeouts:
    request: "4000s"▼

CloudBees CD/RO requirement Status Notes

CloudBees CD/RO requirement	Status	Notes
Upload > 1MB	TrafficPolicy	Max ~4GB (Envoy int32 hard limit; Ingress-NGINX allows 10GB).
Timeout > 60s	TrafficPolicy	Default is 15s (worse than NGF’s 60s).
Session affinity	Broken	GEP-1619 `sessionPersistence` is implemented, but the cookie `Path` is set to the request path instead of `/` — Commander UI still logs out. Bug filed with kgateway.
TCP ports (8443, 61613, 8200)		True TCP passthrough.

Upload > 1MB

TrafficPolicy

Max ~4GB (Envoy int32 hard limit; Ingress-NGINX allows 10GB).

Timeout > 60s

TrafficPolicy

Default is 15s (worse than NGF’s 60s).

Session affinity

Broken

GEP-1619 sessionPersistence is implemented, but the cookie Path is set to the request path instead of / — Commander UI still logs out. Bug filed with kgateway.

TCP ports (8443, 61613, 8200)

True TCP passthrough.

Configure a different gateway controller

If you use a controller not listed above, ensure it provides the following capabilities (derived from CloudBees CD/RO’s current Ingress-NGINX configuration):

Requirement Specification Source

Requirement	Specification	Source
Upload body size	Allow request bodies up to 10GB	`ingress.annotations.proxy-body-size: 10G`
Request timeout	Allow requests up to 4000	`ingress.annotations.proxy-read-timeout: 4000`
Session affinity	Cookie or IP-based sticky sessions for `flow-web`.	`ingress.annotations.affinity: cookie`
TCP passthrough	Byte-level passthrough for ports 8443, 61613, 8200	`ingress-nginx.tcp` ConfigMap entries
TCP idle timeout	10 minutes is sufficient	`ingress-nginx.config.proxy-stream-timeout` (validated: default holds)
Backend protocol	HTTP to `flow-web` (TLS terminates at gateway)	`ingress.annotations.secure-backends: false`

Upload body size

Allow request bodies up to 10GB

ingress.annotations.proxy-body-size: 10G

Request timeout

Allow requests up to 4000

ingress.annotations.proxy-read-timeout: 4000

Session affinity

Cookie or IP-based sticky sessions for flow-web.

ingress.annotations.affinity: cookie

TCP passthrough

Byte-level passthrough for ports 8443, 61613, 8200

ingress-nginx.tcp ConfigMap entries

TCP idle timeout

10 minutes is sufficient

ingress-nginx.config.proxy-stream-timeout (validated: default holds)

Backend protocol

HTTP to flow-web (TLS terminates at gateway)

ingress.annotations.secure-backends: false

Deployment methods

Following are the methods to deploy Kubernetes Gateway API:

Method 1: Use this method when setting up a new cluster or when Ingress-NGINX is not yet deployed. With these values, the chart creates 1 HTTPRoute and 3 TLSRoutes by default. To use TCPRoute on any port instead, refer to Switch a port to TCPRoute.
```
ingress:
  enabled: false
ingress-nginx:
  enabled: false
gatewayApi:
  enabled: true
  gatewayName: "flow-gateway" #Your actual gateway name
  host: "cdro.example.com"     # Your web host▼
```
Method 2: Follow these steps to migrate from an existing Ingress-NGINX deployment:
1. Upgrade the Helm repository first to get the Gateway API values, using the following command:
  helm repo update
  ▼
2. Enable both Ingress and Gateway API simultaneously. Both route traffic to the same backend. Use weighted DNS to split traffic between the Ingress-NGINX IP address and the Gateway IP address.
  # Enable both — coexistence mode ingress: enabled: true host: "cdro.example.com" ingress-nginx: enabled: true # subject to your current value; you may not be using the bundled ingress-nginx subchart gatewayApi: enabled: true gatewayName: "flow-gateway" host: "cdro.example.com" # can be same or different hostname
  ▼
3. Deploy coexistence config above and apply required Gateway policies.
4. Set DNS to route a small percentage of traffic to the Gateway IP (for example, 10 percent).
5. Validate all scenarios (uploads, long operations, Commander UI sessions, ectool).
6. Gradually increase Gateway traffic weight to 100 percent.
7. In a future Helm upgrade, disable ingress.
  ingress: enabled: false ingress-nginx: enabled: false
  ▼