Networking and Ingress

Network configuration and ingress management are critical for exposing CloudBees CD/RO services and controlling traffic flow in Kubernetes deployments. This page explains how to configure load balancing, manage ingress controllers, implement network policies, and transition to the Kubernetes Gateway API. These configurations enable you to secure communication between CloudBees CD/RO components and external clients while maintaining compliance with your organization’s networking standards.

Configure load balancing with ingress

For the default CloudBees CD/RO Helm charts, refer to Ingress values. You can find these cloudbees-flow values referenced in the values.yaml under the Flow ingress configuration section.

An ingress controller typically does not eliminate the need for an external load balancer but adds a layer of routing and control behind the load balancer. However, you can configure the ingress controller so all service endpoints, such as web, server, and repository, may be exposed from the same domain name and load balancer endpoint.

CloudBees recommends configuring your ingress controller so all CloudBees CD/RO services can be exposed through a single load balancer.

By default, ingress is enabled in the CloudBees CD/RO chart. The following is a summary of the settings:

To run CloudBees CD/RO, you must use the ingress-nginx controller with the following required settings:

ingress-nginx.enabled=true
ingress.class=nginx

Parameter Default Value

Parameter	Default Value
ingress-nginx.controller.ingressClass	Default: `flow-ingress`
ingress-nginx.controller.publishService.enabled	Default: `true`
ingress-nginx.controller.scope.enabled	Default: `true`
ingress-nginx.enabled	Default: `true`
ingress-nginx.tcp.61613	CloudBees CD/RO server Default: `{{ .Release.Namespace }}/flow-server:61613`
ingress-nginx.tcp.8200	CloudBees CD/RO repository Default: `{{ .Release.Namespace }}/flow-repository:8200`
ingress-nginx.tcp.8443	CloudBees CD/RO web server Default: `{{ .Release.Namespace }}/flow-server:8443`
ingress-nginx.tcp.9201	CloudBees Analytics database Default: `{{ .Release.Namespace }}/flow-analytics:9201`

ingress-nginx.controller.ingressClass

Default: flow-ingress

ingress-nginx.controller.publishService.enabled

Default: true

ingress-nginx.controller.scope.enabled

Default: true

ingress-nginx.enabled

Default: true

ingress-nginx.tcp.61613

CloudBees CD/RO server

Default: {{ .Release.Namespace }}/flow-server:61613

ingress-nginx.tcp.8200

CloudBees CD/RO repository

Default: {{ .Release.Namespace }}/flow-repository:8200

ingress-nginx.tcp.8443

CloudBees CD/RO web server

Default: {{ .Release.Namespace }}/flow-server:8443

ingress-nginx.tcp.9201

CloudBees Analytics database

Default: {{ .Release.Namespace }}/flow-analytics:9201

Install custom ingress controllers with CloudBees CD/RO

To configure CloudBees CD/RO to use your custom or existing ingress controller:

Configure your ingress controller manifest to expose the following required TCP ports:

TCP port 8200 is required by flow-repository.
TCP ports 8443 and 61613 are required by flow-server.

(Optional) If you are using CloudBees Analytics, TCP port 9201 is required by flow-analytics.

If you need help configuring the ingress controller manifest, refer to Exposing TCP and UDP services in the NGINX Ingress Controller documentation. Ensure you follow the example for TCP/UDP proxy support.

Update your myvalues.yaml with your custom ingress controller values:
```
ingress:
  enabled: true
  host: <your-host>
  annotations: <your-annotations>
  class: <your-custom-ingress-class>▼
```
For a reference, refer to CloudBees CD/RO Helm chart ingress example.
In your myvalues.yaml, ensure ingress-nginx.enabled is configured as false for your installation.
```
ingress-nginx:
  enabled: false▼
```
If you do not have the ingress-nginx.enabled entry in your myvalues.yaml, by default, it is set to false in the cloudbees-flow values file.
If you have not already, apply your ingress controller manifest to your Kubernetes CloudBees CD/RO cluster.

To apply the custom ingress controller configuration, update your CloudBees CD/RO installation:

helm upgrade <server-releasename> cloudbees/cloudbees-flow \
-f <myvalues.yaml> \
--namespace <server-namespace> \
--timeout 10000s▼

Override network policies for CloudBees CD/RO on Kubernetes

As of v2023.04.0, you can override network polices for CloudBees CD/RO on Kubernetes by adding custom network polices and ingress controllers to override the default network policies. This is useful to deploy custom ingress controllers for internal networking for CloudBees CD/RO components or agents.

CloudBees strongly suggests testing all network policy changes in a development environment before releasing them into your production environment.

Before you start

Before overriding the default CloudBees CD/RO network policies:

You must have a NetworkPolicy manifest(s) developed that describes network behavior for all CloudBees CD/RO components.

Example NetworkPolicy manifest

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  annotations:
    meta.helm.sh/release-name: cbflow
    meta.helm.sh/release-namespace: cbflow
  labels:
    app.kubernetes.io/managed-by: Helm
  name: web-policy
  namespace: cbflow
spec:
  ingress:
  - from:
    - ipBlock:
        cidr: 0.0.0.0/0
    - podSelector:
        matchLabels:
          app: flow-bound-agent
          release: cbflow
    - podSelector:
        matchLabels:
          app: flow-server
          release: cbflow
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: kube-system
      podSelector:
        matchLabels:
          app.kubernetes.io/name: ingress-nginx
    ports:
    - port: 2080
      protocol: TCP
    - port: 2443
      protocol: TCP
  podSelector:
    matchLabels:
      app: flow-web
      release: cbflow
  policyTypes:
  - Ingress▼

Your NetworkPolicy manifest(s) must be present in your cluster. To add manifest(s) to your cluster, run the following command:
```
kubectl apply -f <your-NetworkPolicy-manifest.yaml> \
      --namespace <development-server-namespace>▼
```
If you have more than one manifest, use additional instances of -f to include them in the previous command.

If your myvalues.yaml is based on the cloudbees-flow values file v2.23.1 or earlier, you need to update the Network isolation configuration section with new configurations. To do so:

Navigate to Sonatype Nexus and download the latest cloudbees-flow release.
Extract the package, and open the cloudbees-flow/values.yaml.

Find the Network isolation configuration section in the values.yaml and copy it to your myvalues.yaml. Ensure the networkIsolation.networkPolicies configuration is present.

When updating your myvalues.yaml with the latest Network isolation configuration section, ensure:

You do not overwrite any custom networkIsolation.* settings.
Your values file is valid and the indention is correct.

Failing to do so may cause your upgrade to fail or other unexpected behavior.

Override CloudBees CD/RO Kubernetes network policies

After you’ve met the requirements in Before you start, perform the following in the myvalues.yaml:

Navigate to the Network isolation configuration section.

To disable the default CloudBees CD/RO network, configure networkIsolation.networkPolicies.enable: false.

After configuring networkIsolation.networkPolicies.enable: false, CloudBees CD/RO will not create its default network policies, and depend completely on your custom policies. Failure to correctly configure your custom network policies may cause unexpected behavior in your deployment.

In the additionalIngressSelector for each CloudBees CD/RO component, provide the network policy configurations for your ingress.

Example custom cbflow-web ingress configuration

  web:
    additionalIngressSelector:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: kube-system
      podSelector:
        matchLabels:
          app.kubernetes.io/name: ingress-nginx▼

Once you’ve updated your myvalues.yaml, update your flow-server deployment:

helm upgrade <development-server-releasename> cloudbees/cloudbees-flow \
      -f <myvalues.yaml> \
      --namespace <development-server-namespace> \
      --timeout 10000s▼

You can now test your custom network policies in your development environment. After testing is complete, to updated your production servers, refer to Install CloudBees CD/RO within Kubernetes.

Configure ingress-nginx default namespace-specific

CloudBees does not suggest installing multiple instances of CloudBees CD/RO in different namespaces within the same Kubernetes cluster, especially in production environments. This can create multiple naming conflicts among components.

Within your non-production environment, if you must install multiple CloudBees CD/RO instances within different namespaces on the same Kubernetes cluster, by default, this installation fails because of ingress-ngnix naming conflicts. This is caused by each instance using the default naming convention within the flow-server values file instead of a unique name required by each ingress-nginx.controller.ingressClassResource.name instance within a Kubernetes cluster.

An option to fix this error is to use the --set directive as part of the helm install command to provide a unique name for ingress-nginx.controller.ingressClassResource.name in each CloudBees CD/RO instance. To do so, you can adapt the following code sample to your project-specific needs when installing or upgrading a CloudBees CD/RO instance in your non-production environment:

RELEASE="<YOUR-CD/RO-RELEASE>"
NAMESPACE="<YOUR-CD/RO-NAMESPACE>"

helm install $RELEASE cloudbees/cloudbees-flow \
--values=myvalues.yaml \
--namespace $NAMESPACE \
--set ingress-nginx.controller.ingressClassResource.name=$RELEASE.$NAMESPACE▼

Resolve an ingress class name conflict

If you have an ingress class name conflict during your upgrade:

Update the default ingress class name for the ingress-nginx controller with the additional values below in your myvalues.yaml.

ingress:
  enabled: true
  host: <your-host.example.com>
  class: <ingress-class-name>

ingress-nginx:
  enabled: true
  controller:
    ingressClassResource:
      name: <ingress-class-name>▼

After you update the class name, patch the existing ingress by running the following command:

kubectl patch ingress/flow-ingress -p '{"spec": {"ingressClassName":"<ingress-class-name>" }}' -n <namespace>▼

If this does not resolve the naming conflict, ensure the ingress-nginx.ingressClassResource.name tag matches the ingress.class and both are correct. If you update the myvalues.yaml to correct the class names, run the previous command again to resolve the conflict.

Resolve DNS/SSL issues after resolving ingress class name conflicts

If you have a DNS/SSL issue after resolving the ingress class name conflict, restart your ingress-nginx pod to mount the local certificates:

Get the deployment of the ingress-nginx:

kubectl get deployments -n <namespace> | grep ingress-nginx▼

This returns the ingress-nginx deployment, such as:

<your-deployment>-ingress-nginx-controller   1/1     1            1           56m▼

Restart the ingress-nginx deployment:

kubectl rollout restart deployment <your-deployment>-ingress-nginx-controller -n <namespace>▼

The upgrade is complete.

Configure Kubernetes Gateway API

Starting from March 2026, Ingress-NGINX will reach end-of-life (EOL) and will no longer be supported. After this date, Ingress-NGINX will no longer receive updates, security patches, or technical support and it will be removed from the CloudBees CD/RO deployments.

What this means:

Ingress-NGINX will cease to function starting with CloudBees CD/RO version 2026.03.0.
Ingress-NGINX may continue to work, but is no longer supported by CloudBees.
No further updates or security patches will be provided.
Technical support for Ingress-NGINX will no longer be available.

CloudBees strongly recommends that all customers begin evaluating and configuring the Kubernetes Gateway API, along with the required controller-specific policies.

Starting with CloudBees CD/RO release 2026.03.0, customers must manage their own ingress or Gateway API controller independently. If you are currently using the Ingress-NGINX controller bundled with cloudbees-flow, check whether ingress-nginx.enabled: true is set in your Helm values. However, in CloudBees CD/RO version 2026.03.0, Ingress-NGINX continues to be the default option while the Gateway API configuration is currently optional.

When configuring Kubernetes Gateway API, test the API components in a non-production environment before enabling them in production. New configuration components such as gateway, listeners, or policies can directly impact the user experience. Before rollout, validate workflows such as file uploads, long-running operations, commander UI sessions, and ectool connectivity to ensure the configuration works as expected. If you have questions or need assistance, please contact CloudBees Support.

Roles and personas

Kubernetes Gateway API refers to role-based ownership model. CloudBees being an application developer, the following are our responsibilities:

Owner Responsibility

Owner	Responsibility
CloudBees CD/RO `cloudbees-flow` chart	The `cloudbees-flow` chart creates only routes such as HTTPRoute and TCPRoutes. It does not create a Gateway, GatewayClass, or any controller-specific policies. You must create and manage these resources.
User (Cluster operator)	GatewayClass, Gateway resource, TLS certificates, controller-specific policies.

CloudBees CD/RO cloudbees-flow chart

The cloudbees-flow chart creates only routes such as HTTPRoute and TCPRoutes. It does not create a Gateway, GatewayClass, or any controller-specific policies. You must create and manage these resources.

User (Cluster operator)

GatewayClass, Gateway resource, TLS certificates, controller-specific policies.

Prerequisites

Following are the prerequisites for configuring Kubernetes Gateway API:

Kubernetes 1.27+
The following Gateway API experimental CustomResourceDefinitions (CRDs) required for TCPRoute v1alpha2:

The following example uses version 1.2.1. Before proceeding, refer to Official Gateway API Installation Guide to ensure you are using the latest version.
```
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.1/experimental-install.yaml▼
```
A Gateway API controller installed and a Gateway resource created. For more information, refer to Supported and tested gateway controllers.
A TLS Secret accessible to the Gateway’s HTTPS listener. The following options are supported:
- Same namespace as CloudBees CD/RO: Reference the secret directly in certificateRefs. No additional configuration is required and you can reuse the existing ingress TLS secret if it already resides in the same namespace.
- Different namespace: Create a ReferenceGrant in the secret’s namespace to allow the Gateway to reference it. For more information, refer to Cross-namespace reference guide.

Key components

CloudBees CD/RO creates the following components:

HTTPRoute: Routes HTTP/HTTPS traffic to flow-web. Supports SDA mode (paths /flow, /commander, /auth, /rest and non-SDA mode (catch-all /). Disable this route for TCP-only deployments by setting gatewayApi.httpRoute.enabled: false.

TCPRoutes (built-in)

Router name Port Backend Default sectionName

Router name	Port	Backend	Default sectionName
`flow-tcproute-secure-server`	8443	`flow-server`	`flow-secure-server`
`flow-tcproute-stomp`	61613	`flow-server`	`flow-stomp`
`flow-tcproute-repository`	8200	`flow-repository`	`flow-repository`

flow-tcproute-secure-server

8443

flow-server

flow-secure-server

flow-tcproute-stomp

61613

flow-server

flow-stomp

flow-tcproute-repository

8200

flow-repository

Additional TCPRoutes: Configures optional components using gatewayApi.tcpRoute.additional[].

Component Port Service

Component	Port	Service
External agent	7800	`gateway-external-agent-flow-agents`
DOIS Elasticsearch	9200	`flow-devopsinsight`
Analytics	9201	`flow-analytics`
Insecure server	8000	`flow-server`

External agent

7800

gateway-external-agent-flow-agents

DOIS Elasticsearch

9200

flow-devopsinsight

Analytics

9201

flow-analytics

Insecure server

8000

flow-server

=== Points to remember Following are some points to note when configuring Gateway API:

TCPRoute API version: Currently v1alpha2 is experimental. When TCPRoute reaches a stable version, CloudBees will update the default value in the chart. If needed, you can update gatewayApi.tcpRoute.apiVersion in your Helm values at any time to match the latest TCPRoute API version available in your cluster.
Session persistence standardization: GEP-1619 introduces a standard sessionPersistence field on HTTPRoute. When implemented correctly, cookie-based session affinity will become portable without requiring controller-specific policies.

Enable Gateway API

Following helm values are the minimum required values to enable Gateway API

Required Helm values

gatewayApi:
  enabled: true
  gatewayName: "<your-gateway-name>"     # REQUIRED — name of the Gateway resource
  host: "cdro.example.com"               # REQUIRED — hostname for routes▼

Full value reference

To see the complete Gateway API default values from the chart, using the following command:

helm repo update
helm show values cloudbees/cloudbees-flow | sed -n '/### Gateway API configuration section/,/### Miscellaneous configuration section/p' | grep -v "Miscellaneous"▼

The following table provides the default Gateway API values and its description:

Value Default Description

Value	Default	Description
`gatewayApi.enabled`	false	`gateway-external-agent-flow-agents`
`gatewayApi.host`	""	`Hostname for routes (required)`
`gatewayApi.gatewayName`	""	Gateway resource name (required)
`gatewayApi.gatewayName`	release namespace	Gateway namespace
`gatewayApi.httpRoute.enabled`	true	Create HTTPRoute
`gatewayApi.httpRoute.hostnames`	`[gatewayApi.host]`	Override hostnames
`gatewayApi.tcpRoute.apiVersion`	gateway.networking.k8s.io/v1alpha2	Update when TCPRoute graduates
`gatewayApi.tcpRoute.server.sectionName`	flow-secure-server	Must match Gateway listener name
`gatewayApi.tcpRoute.stomp.sectionName`	flow-stomp	Must match Gateway listener name
`gatewayApi.tcpRoute.repository.sectionName`	flow-repository	Must match Gateway listener name
`gatewayApi.tcpRoute.additional`	`[]`	Additional TCP routes

gatewayApi.enabled

false

gateway-external-agent-flow-agents

gatewayApi.host

Hostname for routes (required)

gatewayApi.gatewayName

Gateway resource name (required)

gatewayApi.gatewayName

release namespace

Gateway namespace

gatewayApi.httpRoute.enabled

true

Create HTTPRoute

gatewayApi.httpRoute.hostnames

[gatewayApi.host]

Override hostnames

gatewayApi.tcpRoute.apiVersion

gateway.networking.k8s.io/v1alpha2

Update when TCPRoute graduates

gatewayApi.tcpRoute.server.sectionName

flow-secure-server

Must match Gateway listener name

gatewayApi.tcpRoute.stomp.sectionName

flow-stomp

Must match Gateway listener name

gatewayApi.tcpRoute.repository.sectionName

flow-repository

Must match Gateway listener name

gatewayApi.tcpRoute.additional

[]

Additional TCP routes

Gateway resources to create

Your Gateway must have listeners that match the sectionNames expected by the cloudbees-flow chart. Each TCP listener name must exactly match the corresponding sectionName configured in the Helm values.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: <your-gateway-name>
  namespace: <cdro-namespace>
spec:
  gatewayClassName: <your-controller-class>   # for example, istio, nginx, kgateway
  listeners:
    - name: http
      hostname: "cdro.example.com"
      port: 80
      protocol: HTTP
      allowedRoutes:
        namespaces:
          from: Same
    - name: https
      hostname: "cdro.example.com"
      port: 443
      protocol: HTTPS
      tls:
        mode: Terminate
        certificateRefs:
          - name: <tls-secret>
            kind: Secret
      allowedRoutes:
        namespaces:
          from: Same
    - name: flow-secure-server    # matches gatewayApi.tcpRoute.server.sectionName
      port: 8443
      protocol: TCP
      allowedRoutes:
        namespaces:
          from: Same
        kinds:
          - kind: TCPRoute
    - name: flow-stomp            # matches gatewayApi.tcpRoute.stomp.sectionName
      port: 61613
      protocol: TCP
      allowedRoutes:
        namespaces:
          from: Same
        kinds:
          - kind: TCPRoute
    - name: flow-repository       # matches gatewayApi.tcpRoute.repository.sectionName
      port: 8200
      protocol: TCP
      allowedRoutes:
        namespaces:
          from: Same
        kinds:
          - kind: TCPRoute▼

If your Gateway resides in a different namespace than the CloudBees CD/RO deployment, you must create a ReferenceGrant in the Gateway’s namespace to allow routes from the CloudBees CD/RO namespace to attach. This is a core requirement of the Gateway API specification, and all compliant controllers must implement it.

Ingress-NGINX parity requirements

CloudBees CD/RO relies on the following Ingress-NGINX behaviors. Your Gateway API controller and policies must provide equivalent functionality, otherwise a failure might occur. The default behavior varies by controller. For more information, refer to Supported and tested gateway controllers.

Ingress-NGINX annotation Behavior Error Required action

Ingress-NGINX annotation	Behavior	Error	Required action
`proxy-body-size: 10G`	Allow uploads up to 10GB	Depending on the controller’s default limits, large uploads may trigger an HTTP 413 error.	Increase the upload limit greater than 10 GB using your controller’s policy mechanism.
`proxy-read-timeout: 4000`	Allow blocking API calls up to 4000 seconds	If the controller’s default timeout is lower than the operation duration, it triggers an HTTP 504 error.	Set request timeout greater than 4000 seconds using your controller’s policy mechanism.
`affinity: cookie`	Sticky sessions for Commander UI	Commander UI sign out on navigation	Configure session affinity using your controller’s supported mechanism.
`proxy-stream-timeout: 4000`	TCP idle connections stay open.	The controller’s 10-minute default was sufficient during validation. This behavior might vary.	Validate in your environment and configure if idle TCP connections drop unexpectedly.
`secure-backends: false`	Triggers an HTTP error to backend (TLS terminates at gateway).	Default behavior in Gateway API.	No action required.

proxy-body-size: 10G

Allow uploads up to 10GB

Depending on the controller’s default limits, large uploads may trigger an HTTP 413 error.

Increase the upload limit greater than 10 GB using your controller’s policy mechanism.

proxy-read-timeout: 4000

Allow blocking API calls up to 4000 seconds

If the controller’s default timeout is lower than the operation duration, it triggers an HTTP 504 error.

Set request timeout greater than 4000 seconds using your controller’s policy mechanism.

affinity: cookie

Sticky sessions for Commander UI

Commander UI sign out on navigation

Configure session affinity using your controller’s supported mechanism.

proxy-stream-timeout: 4000

TCP idle connections stay open.

The controller’s 10-minute default was sufficient during validation. This behavior might vary.

Validate in your environment and configure if idle TCP connections drop unexpectedly.

secure-backends: false

Triggers an HTTP error to backend (TLS terminates at gateway).

Default behavior in Gateway API.

No action required.

Supported and tested gateway controllers

The following controller configurations reflect what was working at the time of validation. Some features referenced here are still in the experimental channel of the Gateway API spec and may change in future controller releases.

CloudBees CD/RO provides these examples for reference only and does not guarantee that they will work across all versions. You are responsible for configuring and managing the Gateway, GatewayClass, TLS, policies, and all controller-specific settings, including deployment, maintenance, and validation.

CloudBees CD/RO is responsible only for the HTTPRoute and TCPRoute resources created by the cloudbees-flow chart.

Controller	Upload > 1MB	Timeout > 60s	Session Affinity	TCP Passthrough	Policies Required	Production Ready
[Istio]	No policy needed	No policy needed.	DestinationRule (cookie)		1
[NGF]	ClientSettings policy	Snippets policy	UpstreamSettings policy (ip_hash)		3
[Traefik]	No policy needed	No policy needed	Not supported	(8443 needs TLSRoute)	0	No session affinity
[kgateway]	TrafficPolicy (max ~4GB)	TrafficPolicy	Broken (cookie Path bug)		2	No session affinity

Controller

Upload > 1MB

Timeout > 60s

Session Affinity

TCP Passthrough

Policies Required

Production Ready

[Istio]

No policy needed

No policy needed.

DestinationRule (cookie)

[NGF]

ClientSettings policy

Snippets policy

UpstreamSettings policy (ip_hash)

[Traefik]

No policy needed

Not supported

(8443 needs TLSRoute)

No session affinity

[kgateway]

TrafficPolicy (max ~4GB)

TrafficPolicy

Broken (cookie Path bug)

No session affinity

Istio: Fully support with one policy required

Istio is Envoy-based. During CloudBees' validation, Envoy did not enforce a default upload body size limit, and Istio had request timeouts disabled by default. Based on this behavior, only session affinity required additional policy configuration. Verify that these defaults apply to your specific Istio version. For more information, refer to Istio documentation and Istio Gateway API guide.

Perform the following additional steps for CloudBees CD/RO deployment:

Enable alpha Gateway API features (required for TCPRoute) using the following command:

helm upgrade istiod istio/istiod -n istio-system \
  --set pilot.env.PILOT_ENABLE_ALPHA_GATEWAY_API=true \
  --reuse-values▼

Configure the required policy for session affinity, using the following command:

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: flow-web-sticky
  namespace: <cdro-namespace>
spec:
  host: flow-web.<cdro-namespace>.svc.cluster.local
  trafficPolicy:
    loadBalancer:
      consistentHash:
        httpCookie:
          name: ROUTE
          path: /
          ttl: 3600s▼

The following table gives CloudBees CD/RO requirements for configuring Istio:

CloudBees CD/RO requirement	Status	Notes
Upload >1MB	No policy needed	Envoy has no default body size limit.
Timeout >60s	No policy needed	Istio disables request timeout by default
Session affinity	DestinationRule	Cookie-based, functionally equivalent to Ingress-NGINX
TCP ports (8443, 61613, 8200)		TCP ports (8443, 61613, 8200)

CloudBees CD/RO requirement

Status

Notes

Upload >1MB

No policy needed

Envoy has no default body size limit.

Timeout >60s

No policy needed

Istio disables request timeout by default

Session affinity

DestinationRule

Cookie-based, functionally equivalent to Ingress-NGINX

TCP ports (8443, 61613, 8200)

NGINX Gateway Fabric (NGF): Fully supported with limitation in OSS version

For NGF Helm installation, refer to Installing NGF.

Perform the following additional steps for CloudBees CD/RO deployment:

Enable experimental features and SnippetsPolicy, using the following command:

helm upgrade ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric \
  --namespace nginx-gateway \
  --set nginxGateway.gwAPIExperimentalFeatures.enable=true \
  --set nginxGateway.snippets.enable=true \
  --reuse-values▼

Apply the three required policies, using the following command:

# Policy 1: Upload size (replaces proxy-body-size: 10G)
apiVersion: gateway.nginx.org/v1alpha1
kind: ClientSettingsPolicy
metadata:
  name: flow-client-settings
  namespace: <cdro-namespace>
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: HTTPRoute
    name: flow-httproute
  body:
    maxSize: "10g"
---
# Policy 2: Request timeout (replaces proxy-read-timeout: 4000)
# Must target Gateway (not HTTPRoute) — CRD constraint
apiVersion: gateway.nginx.org/v1alpha1
kind: SnippetsPolicy
metadata:
  name: flow-proxy-timeouts
  namespace: <cdro-namespace>
spec:
  targetRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: <your-gateway-name>
  snippets:
    - context: http.server.location
      value: |
        proxy_read_timeout 4000s;
        proxy_send_timeout 4000s;
---
# Policy 3: Session affinity (replaces affinity: cookie)
# Note: ip_hash — not cookie-based. All requests from same client IP → same web server pod.
# Acceptable for enterprise deployments; may cause uneven load if clients share NAT IP.
apiVersion: gateway.nginx.org/v1alpha1
kind: UpstreamSettingsPolicy
metadata:
  name: flow-upstream-settings
  namespace: <cdro-namespace>
spec:
  targetRefs:
    - group: ""
      kind: Service
      name: flow-web
  loadBalancingMethod: "ip_hash"▼

The following table gives CloudBees CD/RO requirements for configuring NGF:

CloudBees CD/RO requirement Status Notes

CloudBees CD/RO requirement	Status	Notes
Upload >1MB	ClientSettingsPolicy	`body.maxSize: "10g"`
Timeout >60s	SnippetsPolicy	Raw NGINX config injection; SnippetsPolicy must be explicitly enabled
Session affinity	UpstreamSettingsPolicy	`ip_hash` - IP-based, not cookie-based
TCP ports (8443, 61613, 8200)		True TCP passthrough

Upload >1MB

ClientSettingsPolicy

body.maxSize: "10g"

Timeout >60s

SnippetsPolicy

Raw NGINX config injection; SnippetsPolicy must be explicitly enabled

Session affinity

UpstreamSettingsPolicy

ip_hash - IP-based, not cookie-based

TCP ports (8443, 61613, 8200)

True TCP passthrough

Allows raw NGINX config injection. Disabled by default. Evaluate whether this is acceptable for your security posture. NGINX Plus alternative: native timeout policies without raw injection.

Following are the known limitations and considerations for Open Source (OSS) Controllers:

Limitation Impact Alternative

Limitation	Impact	Alternative
Cookie-based sticky sessions require NGINX Plus	OSS uses `ip_hash` for all requests from the same client IP to go to the same pod. Clients behind shared NAT may cluster on one pod causing uneven load.	NGINX Gateway Fabric Plus (native `sticky cookie`), Istio (`DestinationRule`), or accept `ip_hash trade-offs` for enterprise deployments.
`SnippetsPolicy` allows raw NGINX config injection	Security risk in multi-tenant clusters; must be explicitly enabled.	NGINX Gateway Fabric Plus has native timeout policies without raw injection.
Policies are NGF-specific CRDs	`ClientSettingsPolicy`, `SnippetsPolicy`, `UpstreamSettingsPolicy` are not portable to other controllers	Each controller has its own policy mechanism.

Cookie-based sticky sessions require NGINX Plus

OSS uses ip_hash for all requests from the same client IP to go to the same pod. Clients behind shared NAT may cluster on one pod causing uneven load.

NGINX Gateway Fabric Plus (native sticky cookie), Istio (DestinationRule), or accept ip_hash trade-offs for enterprise deployments.

SnippetsPolicy allows raw NGINX config injection

Security risk in multi-tenant clusters; must be explicitly enabled.

NGINX Gateway Fabric Plus has native timeout policies without raw injection.

Policies are NGF-specific CRDs

ClientSettingsPolicy, SnippetsPolicy, UpstreamSettingsPolicy are not portable to other controllers

Each controller has its own policy mechanism.

Traefik v3: Limited — session affinity not supported

For more information about installing Traefik, refer to Traefik documentation.

Perform the following additional steps for CloudBees CD/RO deployment:

Configure entryPoints: Define Traefik entryPoints that match the Gateway listener names and ports.
Update Helm values: Customize the Traefik Helm values to include the required entryPoints before creating the Gateway.
Configure TLS passthrough for port 8443: Unlike other controllers that use TCPRoute, Traefik requires protocol: TLS with tls.mode: Passthrough in the Gateway listener, and a TLSRoute (not TCPRoute) for port 8443.

For more information, refer to Traefik Helm Chart.

Traefik requires pre-configured entryPoints that match Gateway listener names and ports — the Traefik Helm values must be customized accordingly before creating the Gateway. Additionally, Traefik’s protocol: TCP listener intercepts TLS connections on port 8443 and uses TLS protocol with tls.mode: Passthrough and a TLSRoute for that port instead of a TCPRoute.

kgateway v2.3: Limited session affinity broken

For more information about installing kgateway v2.3, refer to kgateway documentation.

Perform the following additional steps for CloudBees CD/RO deployment:

Enable experimental features (required for TCPRoute and GEP-1619 session persistence), using the following command:

helm upgrade -i -n kgateway-system kgateway \
  oci://cr.kgateway.dev/kgateway-dev/charts/kgateway \
  --version v2.3.0-main \
  --set controller.extraEnv.KGW_ENABLE_GATEWAY_API_EXPERIMENTAL_FEATURES=true▼

Apply two required policies, using the following command:

# Policy 1: Upload size (max ~4GB due to Envoy int32 limit)
apiVersion: gateway.kgateway.dev/v1alpha1
kind: TrafficPolicy
metadata:
  name: flow-buffer-limits
  namespace: <cdro-namespace>
spec:
  targetRefs:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
      name: flow-httproute
  buffer:
    maxRequestSize: "4095Mi"
---
# Policy 2: Request timeout (Envoy default is 15s — more aggressive than NGF's 60s)
apiVersion: gateway.kgateway.dev/v1alpha1
kind: TrafficPolicy
metadata:
  name: flow-proxy-timeout
  namespace: <cdro-namespace>
spec:
  targetRefs:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
      name: flow-httproute
  timeouts:
    request: "4000s"▼

CloudBees CD/RO requirement	Status	Notes
Upload > 1MB	TrafficPolicy	Max ~4GB (Envoy int32 hard limit - Ingress-NGINX allows 10GB)
Timeout > 60s allows raw NGINX config injection	TrafficPolicy	Default is 15s (worse than NGF’s 60s).
Session affinity	Broken	GEP-1619 sessionPersistence implemented but cookie Path set to request path instead of / — Commander UI still logs out. Bug filed with kgateway.Each controller has its own policy mechanism.
TCP ports (8443, 61613, 8200)		True TCP passthrough

CloudBees CD/RO requirement

Status

Notes

Upload > 1MB

TrafficPolicy

Max ~4GB (Envoy int32 hard limit - Ingress-NGINX allows 10GB)

Timeout > 60s allows raw NGINX config injection

TrafficPolicy

Default is 15s (worse than NGF’s 60s).

Session affinity

Broken

GEP-1619 sessionPersistence implemented but cookie Path set to request path instead of / — Commander UI still logs out. Bug filed with kgateway.Each controller has its own policy mechanism.

TCP ports (8443, 61613, 8200)

True TCP passthrough

Configure a different gateway controller

If you use a controller not listed above, ensure it provides the following capabilities (derived from CloudBees CD/RO’s current Ingress-NGINX configuration):

Requirement Specification Source

Requirement	Specification	Source
Upload body size	Allow request bodies up to 10GB	`ingress.annotations.proxy-body-size: 10G`
Request timeout	Allow requests up to 4000	`ingress.annotations.proxy-read-timeout: 4000`
Session affinity	Cookie or IP-based sticky sessions for `flow-web`.	`ingress.annotations.affinity: cookie`
TCP passthrough	Byte-level passthrough for ports 8443, 61613, 8200	`ingress-nginx.tcp` ConfigMap entries
TCP idle timeout	10 minutes is sufficient	`ingress-nginx.config.proxy-stream-timeout` (validated: default holds)
Backend protocol	HTTP to `flow-web` (TLS terminates at gateway)	`ingress.annotations.secure-backends: false`

Upload body size

Allow request bodies up to 10GB

ingress.annotations.proxy-body-size: 10G

Request timeout

Allow requests up to 4000

ingress.annotations.proxy-read-timeout: 4000

Session affinity

Cookie or IP-based sticky sessions for flow-web.

ingress.annotations.affinity: cookie

TCP passthrough

Byte-level passthrough for ports 8443, 61613, 8200

ingress-nginx.tcp ConfigMap entries

TCP idle timeout

10 minutes is sufficient

ingress-nginx.config.proxy-stream-timeout (validated: default holds)

Backend protocol

HTTP to flow-web (TLS terminates at gateway)

ingress.annotations.secure-backends: false

Deployment methods

Following are the methods to deploy Kubernetes Gateway API:

Method 1: Use this method when setting up a new cluster or when Ingress-NGINX is not yet deployed:

ingress:
  enabled: false
ingress-nginx:
  enabled: false
gatewayApi:
  enabled: true
  gatewayName: "flow-gateway" #Your actual gateway name
  host: "cdro.example.com"     # Your web host▼

Method 2: Follow these steps to migrate from an existing Ingress-NGINX deployment:
1. Upgrade the Helm repository first to get the Gateway API values, using the following command:
  helm repo update
  ▼
2. Enable both Ingress and Gateway API simultaneously. Both route traffic to the same backend. Use weighted DNS to split traffic between the Ingress-NGINX IP address and the Gateway IP address.
  # Enable both — coexistence mode ingress: enabled: true host: "cdro.example.com" ingress-nginx: enabled: true # subject to you current value. you may not use our internal ingress-nginx subchart gatewayApi: enabled: true gatewayName: "flow-gateway" host: "cdro.example.com" # can be same or different hostname
  ▼
3. Deploy coexistence config above and apply required Gateway policies.
4. Set DNS to route a small percentage of traffic to the Gateway IP (for example, 10 percent).
5. Validate all scenarios (uploads, long operations, Commander UI sessions, ectool).
6. Gradually increase Gateway traffic weight to 100 percent.
7. In a future Helm upgrade, disable ingress.
  ingress: enabled: false ingress-nginx: enabled: false
  ▼