Installing CloudBees CD on Kubernetes

This section describes installing CloudBees CD on Kubernetes. These instructions assume you are familiar with Kubernetes concepts and the tooling required for establishing a Kubernetes cluster.

CloudBees CD on Kubernetes topology

The following diagram depicts a clustered topology for a production CloudBees CD ecosystem running within one namespace in a Kubernetes cluster.

k8s topology

This diagram shows three pods, whose deployment is controlled by three corresponding Helm charts:

  • cloudbees-flow: This chart installs the CloudBees CD server, the CloudBees CD web server, the repository server, the DevOps Insight server and a bound agent. The bound agent serves as a local agent for the CloudBees CD server, the web server and the repository server.

  • cloudbees-flow-agent: Provides configuration for one or more agents to execute commands, monitor status, and collect results, in parallel across the pods. Each agent can be configured independently to provide specific resources for the task at hand.

Prerequisites

  • Kubernetes cluster and Helm: see Kubernetes cluster requirements.

  • Database instance

    • For a production environment installation, see Database instance for details.

    • For a non-production environment installation, the CloudBees CD Helm chart automatically installs a MariaDB database instance as a built-in database.

Non-production environment

Cluster capacity

The table below specifies the default memory and CPU requested for each CloudBees CD component in a non-production environment.

CloudBees CD component/service CPU Memory

CloudBees CD Server

Per replica: 1.5 CPU, Default number of replicas: 1

Per replica: 6 GiB (Memory assigned to CloudBees CD JVM: 4 GiB)

Web Server

0.25 CPU

256 MiB

DevOps Insight Server

Per replica: 0.1 CPU, Default number of replicas: 1

Per replica: 2 GiB (Memory assigned to Elasticsearch JVM: 1 GiB)

Repository Server

0.25 CPU

512 MiB (Memory assigned to Repository JVM: 512 MiB)

CloudBees CD Agent (Bound)

0.25 CPU

512 MiB (Memory assigned to Agent JVM: 256 MiB)

Storage

Consult the table below for default storage requirements for a non-production environment.

CloudBees CD component/service Storage type Storage amount

CloudBees CD Server

ReadWriteOnce

5 GiB

DevOps Insight Server

ReadWriteOnce

10 GiB

Repository Server

ReadWriteOnce

10 GiB

Production environment

Cluster capacity

For current cluster capaccity and sizing specifications, see Kubernetes cluster requirements.

Separate database instance

You must configure a separate database for your installation of CloudBees CD before you initiate the installation. For current database server and sizing specifications, see Database server specifications.

Database configuration

The database can be installed inside or outside the cluster. Update the database section in the cloudbees-cd-defaults.yaml file that is specified when installing the cloudbees-flow chart. See the Database Values section in Configuration Values.

Persistent Storage

CloudBees CD components need varying levels of persistent storage.

  • Non-shared storage: A disk or file system that cannot be shared between two pods or two nodes. Examples include AWS EBS, GCP Persistent Disk, and Azure Disk.

  • Shared storage: A disk or file system that can be shared between two pods or two nodes. Examples include AWS EFS, GCP NFS Server, and Azure Filestore.

Consult the table below for storage requirements.

CloudBees CD component/service Storage type Storage amount Mount point

Shared storage

CloudBees CD Server

CloudBees CD Agent (bound)

ReadWriteMany

5 GiB

/plugin-data

Repository Server

ReadWriteMany

20 GiB

/repository-data

Non-shared storage

DevOps Insight Server

ReadWriteOnce

Per replica: 4, 10 GiB Default number of replicas: 3

/elasticsearch-data

CloudBees CD Agent (worker)

ReadWriteMany

5GiB

/workspace

Zookeeper

/var/lib/zookeeper

CloudBees CD server persistent volume provisioning

Update the storage section in the cloudbees-cd-defaults.yaml file specified when installing the cloudbees-flow chart. See Storage values for details.

CloudBees CD agent persistent volume provisioning

Update the storage section in the cloudbees-cd-agent-defaults.yaml file specified when installing the cloudbees-flow-agent chart. See Storage values for details.

Creating persistent shared storage

Create persistent shared storage on Google Cloud Platform or Amazon EKS. To get started, clone the cloudbees-example repo and then use the follow instructions.

  • Google Cloud Platform: Use Filestore to create a fully-managed NFS file server on Google Cloud.

    Find information and scripts to provision a Google Cloud Filestore instance in the cloudbees-examples repo.

  • Amazon EKI: Use Amazon Elastic File System (EFS) to create fully-managed persistent storage.

    Find information and scripts to provision an EFS instance in the cloudbees-examples repo.

CloudBees CD server installation and upgrade

Overview

This option installs the CloudBees CD server, built-in database, web server, and repository server into one cluster. A local bound agent and CloudBees CD tools are also installed.

In a production configuration, connect CloudBees CD to a separate database. If CloudBees CD is initially installed with the built-in database, you can reconfigure it to use a separate database at any time. See Separate database instance for further details.

Using a separate database requires a CloudBees CD enterprise license. You must configure an external database at the same time as you install your enterprise license to prevent error messages about an unsupported configuration or a license requirement.

Installation

Create your custom values file to override Helm chart values for your installation. For your convenience, preconfigured values files are available here to get you started. Use these files as is, or use them as a starting point.

Best practice is to make a local copy of one of the preconfigured files and modify that local copy with your settings, as needed. That way, you have a record of your configuration when it comes time to update your installation.
Values file Decription

cloudbees-cd-demo.yaml

File for use with non-production installations. It configures a single node CloudBees CD server installation with a non-production license and installs MariaDB as the database.

cloudbes-cd-prod.yaml

File for use with production installations. You must configure your database, storage and CloudBees CD credentials in a local copy of this file before it can be used.

cloudbees-cd-defaults.yaml

File listing all Helm chart values along with their default value. Use as a reference when specifying additional configuration in your local values file.

Add public CloudBees repository as follows:

helm repo add cloudbees https://charts.cloudbees.com/public/cloudbees
helm repo update

Run the following Helm command to install CloudBees CD.

helm install releaseName cloudbees/cloudbees-flow \
          -f valuesFile --namespace nameSpace --timeout 10000s
helm upgrade releaseName cloudbees/cloudbees-flow \
          -f valuesFile --namespace nameSpace --timeout 10000s

where:

releaseName

A user-defined name for this particular installation. If not specified, one is generated for you.

valuesFile

The local YAML-formatted values file with your specific chart override values. Specify something like my-values.yaml. If not specified, default chart values are used, as defined in the cloudbees-example repo.

nameSpace

A user-defined name for the namespace under which CloudBees CD components are installed.

CloudBees CD installation takes several minutes. Once all the pods are running, you can track the progress of the CloudBees CD server initialization,via the flow-server-init-job job log file in the Kubernetes dashboard.

CloudBees CD agent installation and upgrade

Overview

This option installs the CloudBees CD agent, only. This allows for configuring multiple agents separately to fit your particular needs. You must have already installed the CloudBees CD server in the cluster. If you need to install the CloudBees CD server, see CloudBees CD server installation and upgrade before proceeding with the agent installation below.

Installation

Create your custom values file to override Helm chart values to use for installing the agents. For your convenience, preconfigured values files are available here to get you started. Use these files as is, or use them as a starting point.

Best practice is to make a local copy of one of the preconfigured files and modify that local copy with your settings, as needed. That way, you have a record of your configuration when it comes time to update your installation.

When multiple replicas are configured in the values file, via the replicas value, pay particular attention to the way in which the agent resource name template is defined so that all replicas are registered properly. See Agent resource name templates for complete information.
Values file Decription

cloudbees-cd-agent-example.yaml

Example file to use for an agent installation. You must configure your agent settings in a local copy of this file before it can be used.

cloudbees-cd-agent-defaults.yaml

File listing all Helm chart values along with their default value. Use as a reference when specifying additional configuration in your local values file. cloudbees-flow-agent chart configuration values for details about values in this file.

Add public CloudBees repository as follows:

helm repo add cloudbees https://charts.cloudbees.com/public/cloudbees
helm repo update

Run the following Helm command to install or upgrade.

Before initiating an install operation, you must create a namespace, as follows.

kubectl create namespace 

Then proceed with the installation.

helm install  cloudbees/cloudbees-flow-agent \
      -f  --namespace  --timeout 10000 \
      [--set-file customScript=]

helm upgrade cloudbees/cloudbees-flow-agent \
      -f  --namespace  --timeout 10000 \
      [--set-file customScript=]

where:

agentReleaseName

A user-defined name for this particular installation. If not specified, one is generated for you.

agentValuesFile

The local YAML-formatted values file with your specific chart override values. Specify something like my-values.yaml. If not specified, default chart values are used, as defined in the cloudbees-example repo.

nameSpace

A user-defined name for the namespace under which the CloudBees agent is installed. Specify the same name used for CloudBees CD server namespace.

custom-script

A user-defined script to be installed in the agent container. See Configuring third-party tools.

Configuring third-party tools

CloudBees CD agents typically do work using third-party tools installed on or accessible to the agent. In a traditional deployment, this typically means that the tools are installed locally on the agent machine. For agents running in a Kubernetes cluster, third-party tools can be deployed in agent container so they are immediately accessible by the agent. Configuration can be accomplished in one of the following ways: making a custom image or dynamically configuring the tools .

Making a custom container

For installations where the set of third-party tools is well defined and static, create your own container using the the CloudBees CD agent image as the base image to which the third-party tools are added. Below is an example Dockerfile configuring the custom container.

FROM cloudbees/cbflow-agent:9.2.0.139827_1.0.7_20191110

RUN apt-get update && \
apt-get install git apt-utils apt-transport-https wget curl gnupg gnupg2 gnupg1 -y
apt-get update && curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | \
       apt-key add -  && \
       echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" | \
       tee -a /etc/apt/sources.list.d/kubernetes.list
curl -L https://git.io/get_helm.sh | bash

Creating a custom script

For installations needing a more flexible method, where the set of tools may vary, use the --set-file option to the helm command to load the installation script into the container. The example custom script below contains commands to install git in the pod using the apt-get package manager.

apt-get update
apt-get install git apt-utils apt-transport-https wget curl gnupg gnupg2 gnupg1 -y
apt-get update && curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | \
       apt-key add -  && \
       echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" | \
       tee -a /etc/apt/sources.list.d/kubernetes.list
curl -L https://git.io/get_helm.sh | bash

Load Balancing with Ingress

This CloudBees CD chart provides support for load balancing with the NGINIX Ingress controller, which allows simple host and URL based HTTP routing. Note that an Ingress controller typically doesn’t eliminate the need for an external load balancer—it simply adds an additional layer of routing and control behind the load balancer. With Ingress configured, all service endpoints like web, server, repository are exposed from same domain name and load-balancer endpoint.

As a best practice, configure the Ingress controller so that all CloudBees CD services can be exposed through a single load balancer. By default, Ingress is enabled in the CloudBees CD chart. Following is a summary of the settings:

nginx-ingress.controller.ingressClass

Default: flow-ingress

nginx-ingress.controller.publishService.enabled

Default: true

nginx-ingress.controller.scope.enabled

Default: true

nginx-ingress.enabled

Default: true

nginx-ingress.tcp.61613

CloudBees CD server

Default: {{ .Release.Namespace }}/flow-server:61613

nginx-ingress.tcp.8200

CloudBees CD repository

Default: {{ .Release.Namespace }}/flow-repository:8200

nginx-ingress.tcp.8443

CloudBees CD web server

Default: {{ .Release.Namespace }}/flow-server:8443

nginx-ingress.tcp.9200

DevOps Insight Elasticsearch database

Default: {{ .Release.Namespace }}/flow-devopsinsight:9200

nginx-ingress.tcp.9500

DevOps Insight server

Default: {{ .Release.Namespace }}/flow-devopsinsight:9500

Configuration Values

Helm provides several ways to set value fields. The CloudBees CD server and agent Helm charts provide CloudBees’s default values. Helm stores these values in the chart’s cloudbees-cd-defaults.yaml file found here .

You can override these fields as follows:

  • On the helm install command line using the --set parameter.

  • In a local myvalues.yaml file.

CloudBees recommends using a local values file for your installation to keep all of your CloudBees CD environment settings.

cloudbees-flow chart configuration values

The following table lists common configurable parameters of the cloudbees-flow chart and their default values. See the chart’s cloudbees-cd-defaults.yaml for the exhaustive list of options.

Name Description/Default

Bound agent values

boundAgent.ecconfigure

String is interpreted as if passed to ecconfigure utility within the container

Default: --agentInitMemoryMB=256 --agentMaxMemoryMB=256

boundAgent.imageName

Default: cbflow-agent

boundAgent.logLevel

Default: DEBUG

boundAgent.replicas

Default: 1

boundAgent.resources.limits.cpu

Default: 0.25

boundAgent.resources.limits.memory

Default: 1024MiB

boundAgent.resources.requests.cpu

Default: 0.25

boundAgent.resources.requests.memory

Default: 512MiB

Database values

database.clusterEndpoint

Use this option if your database is residing in the same Kubernetes cluster as Flow. Notation is db-service.namespace If deploying into the same namespace, .namespace component can be omitted.

Default: null

database.externalEndpoint

Database endpoint, database or schema , principal that has full read/write access on that schema

Default: null

database.dbName

Default: null

database.dbPassword

Default: null

database.dbPort

Default: null

database.dbType

The database type with which CloudBees CD persistence works. One of:

mysql —MYSQL

mariadb —MariaDB

sqlserver —MS SQLServer

oracle —Oracle

Default: null

database.dbUser

Default: null

database.existingSecret

Use this option if you have or are planning to deploy the credential’s secret yourself. The layout has to be the same as that of server-secrets.yaml::dbSecret

Default: null

demoMode

Default: false

DevOps Insight server values

dois.credentials.adminPassword dois.credentials.existingSecret

Credentials for administrative access to Elasticsearch data. It sets a password for the admin ` user with the corresponding rights. If the password is an empty string, then user `admin is not created and administrative access is not granted.

Default: null

dois.enabled

Flag that dictates whether this workload and its accompanying services are to be installed

Default: true

dois.esClusterName

Elasticsearch cluster name

Default: null

dois.esMinMasterNodes

Minimum number of master-eligible nodes that must be visible in order to form an Elasticsearch cluster.

Default: 1

dois.esNumberOfShards

The number of primary shards that an index must have.

Default: null

dois.esRam

Elasticsearch heap size in MB.

Default: 2048

dois.expose

Expose DevOps Insght as externally available services ingress needs to ports 9200 and 9500 to be open to the outer world.

Default: true

dois.imageName

Default: cbflow-dois

dois.lsInitRam

Default: 512

dois.lsMaxRam

Default: 1024

dois.replicas

Number of Elasticsearch nodes

Default: 1

dois.resources.limits.cpu

Default: 1

dois.resources.limits.memory

Default: 3.5GiB

dois.resources.requests.cpu

Default: 0.1

dois.resources.requests.memory

Default: 3.5GiB

dois.serverEndpoint

CloudBees CD server endpoint. If installing in different namespace than server server name has to be stated in form service-name.namespace . Default: flow-server

dois.serviceEndpoint

The DevOps Insigt server service endpoint to be configured on the remote CloudBees CD server. It is assumed the remote CloudBees CD server is located in the same Kubernetes cluster.

Default: flow-devopsinsight.{{ .Release.Namespace }}

CloudBees CD credential values

flowCredentials.adminPassword

Default: changeme

flowCredentials.existingSecret

Either specify the secret where the admin user password is stored under the CBF_SERVER_ADMIN_PASS key, recommended for production, or specify the adminPassword .

Default: null

CloudBees CD server license values (optional)

flowLicense.existingSecret

Default: null

flowLicense.licenseData

Default: null

Images values

images.pullPolicy

Default: IfNotPresent

images.repository

Default: 547883162893.dkr.ecr.us-east-1.amazonaws.com

images.tag

Default: 9.2.0

Network isolation values

networkIsolation.allowFromCidr

The CIDR allowed to make connection to all exposed CloudBees CD endpoints.

Default: 0.0.0.0/0

Nginx-ingress values

nginx-ingress.controller.ingressClass

Default: flow-ingress

nginx-ingress.controller.publishService.enabled

Default: true

nginx-ingress.controller.scope.enabled

Default: true

nginx-ingress.enabled

Default: true

nginx-ingress.tcp.61613

Default: {{ .Release.Namespace }}/flow-server:61613

nginx-ingress.tcp.8200

Default: {{ .Release.Namespace }}/flow-repository:8200

nginx-ingress.tcp.8443

Default: {{ .Release.Namespace }}/flow-server:8443

nginx-ingress.tcp.9200

Default: {{ .Release.Namespace }}/flow-devopsinsight:9200

nginx-ingress.tcp.9500

Default: {{ .Release.Namespace }}/flow-devopsinsight:9500

Repository values

repository.ecconfigure

String to pass to `ecconfigure ` utility.

Default: --repositoryInitMemoryMB=256 --repositoryMaxMemoryMB=512

repository.enabled

Flag that dictates whether this workload and its accompanying services are to be installed

Default: true

repository.expose

Expose repository as an externally available service that ingress needs to port port 8200.

Default: true

repository.imageName

Default: cbflow-repository

repository.logLevel

Default: DEBUG

repository.replicas

Default: 1

repository.resources.limits.cpu

Default: 0.25

repository.resources.limits.memory

Default: 1024MiB

repository.resources.requests.cpu

Default: 0.25

repository.resources.requests.memory

Default: 512MiB

repository.zoneName

The zone, assigned to the repository instance, created in the CloudBees CD server to represent this repository instance. The zone must exist in CloudBees CD server instance.

Default: null

CloudBees CD server values

server.ecconfigure

String to pass to `ecconfigure ` utility.

Default: --serverInitMemoryMB=4096 --serverMaxMemoryMB=4096

server.enabled

Flag that dictates whether the web workload and its accompanying services are to be installed.

Default: true

server.expose

As CloudBees CD requires repository to be exposed as externally available services, ingress needs to ports 8443 and 61613 to be open to the outer world.

Default: true

server.imageName

Default: cbflow-server

server.logLevel

Master loglevel for com.electriccloud package

Default: INFO

server.replicas

Default: 1

server.resources.limits.cpu

Default: 4

server.resources.limits.memory

Default: 6GiB

server.resources.requests.cpu

Default: 1.5

server.resources.requests.memory

Default: 6GiB

server.zk.host

Default: zookeeper

server.zk.port

Default: 2181

Server storage values

storage.volumes.doisStorage.accessMode

Default: ReadWriteOnce

storage.volumes.doisStorage.name

Default: elasticsearch-data

storage.volumes.doisStorage.storage

Default: 10GiB

storage.volumes.repositoryStorage.accessMode

Default: ReadWriteOnce

storage.volumes.repositoryStorage.name

Default: flow-repo-artifacts

storage.volumes.repositoryStorage.storage

Default: 20GiB

storage.volumes.repositoryStorage.storageClass

Specify non-platform-default or custom storage class.

Default: null

storage.volumes.serverPlugins.accessMode

Default: ReadWriteMany

storage.volumes.serverPlugins.name

Default: flow-server-shared

storage.volumes.serverPlugins.storage

Default: 5GiB

storage.volumes.serverPlugins.storageClass

Storage class for plugins directory. Currently it has to be shared across all server and web replicas. Has to be ReadWriteMany accessible.

Default: null

CloudBees CD web server values

web.ecconfigure

String to pass to `ecconfigure ` utility.

web.enabled

Flag that dictates whether the web workload and its accompanying services are to be installed.

Default: true

web.imageName

Default: cbflow-web

web.ingress.annotations. kubernetes.io/ingress.class

Default: flow-ingress

web.ingress.annotations.nginx.ingress. kubernetes.io/affinity

Default: cookie

web.ingress.annotations.nginx.ingress. kubernetes.io/proxy-body-size

Default: 10GB

web.ingress.annotations.nginx.ingress. kubernetes.io/secure-backends

Default: false

web.ingress.annotations.nginx.ingress. kubernetes.io/session-cookie-hash

Default: sha1

web.ingress.annotations.nginx.ingress. kubernetes.io/session-cookie-name

Default: route

web.ingress.cert.crt web.ingress.cert.key

Certificates for WEB ingress. Normally should be set when helm install ` executes using: `--set-file web.ingress.cert.key=path/to/key ` `--set-file web.ingress.cert.crt=path/to/crt

This section is just an example that ingress can be configured with certs for TLS.

Default: null

web.ingress.enabled

Default: true

web.ingress.host

Default: fbapp.ecsaas.xyz

web.replicas

Default: 1

web.resources.limits.cpu

Default: 1

web.resources.limits.memory

Default: 512MiB

web.resources.requests.cpu

Default: 0.25

web.resources.requests.memory

Default: 256MiB

web.service.type

Default: ClusterIP

Zookeeper values

zookeeper.enabled

Default: true

zookeeper.fullnameOverride

Default: zookeeper

zookeeper.podLabels.mode

Default: private

zookeeper.podLabels.ownerApp

Default: cloudbees-flow

zookeeper.podLabels.role

Default: cluster-coordinator

zookeeper.replicaCount

Default: 3

zookeeper.resources.limits.cpu

Default: 250m

zookeeper.resources.limits.memory

Default: 512MiB

zookeeper.resources.requests.cpu

Default: 250m

zookeeper.resources.requests.memory

Default: 512MiB

cloudbees-flow-agent chart configuration values

The following table lists common configurable parameters of the cloudbees-flow-agent chart and their default values. See the chart’s values.yaml for the exhaustive list of options.

Name Description/Default

ecconfigure

String to pass to ecconfigure utility.

Default:

--agentInitMemoryMB=16 --agentMaxMemoryMB=64

CloudBees CD credential values

CloudBees CD server credentials to use in order to register with agent as a resource on the CloudBees CD server. The admin user is used by default. You may user a different user if desired. The user must have modify permissions on CloudBees CD server resources.

flowCredentials.existingSecret

Default: null

flowCredentials.password

Default: changeme

flowCredentials.user

Default: admin

Image values

imageName

Default: cbflow-agent

images.pullPolicy

Default: IfNotPresent

images.repository

Default: 547883162893.dkr.ecr.us-east-1.amazonaws.com

images.tag

Default: 9.2.0

Resource values

resources.limits.cpu

Default: 1

resources.limits.memory

Default: 1024Mi

resources.requests.cpu

Default: 0.25

resources.requests.memory

Default: 512Mi

Agent storage values

storage.volumes.agentWorkspace.accessMode

Default: ReadWriteOnce

storage.volumes.agentWorkspace.storageClass

Use with any non platform-default or custom storage class.

Default: null

storage.volumes.agentWorkspace.name

Default: flow-agent-workspace

storage.volumes.agentWorkspace.storage

Default: 5GiB

Miscellaneous agent values

logLevel

Master loglevel for com.electriccloud package

Default: DEBUG

replicas

Default: 1

resourceName

CloudBees CD resource name that agents of this deployment are assigned to. See Agent resource name templates for more options.

Default: hostname

resourcePools

List of resource pools with which agents try to register upon coming up and registering with server.

Default: null

resourceType

Type of resource to create on the remote CloudBees CD server. This argument is relevant only when the CloudBees CD server is using a mixed-mode license (concurrent resources and registered hosts). Valid options for this value are concurrent or registered.

Default: null

serverEndpoint

Default: null

trustedAgent

Set to `true ` to configure the agent as a trusted agent, restricting the agent to one CloudBees CD server deployment.

Default: false

workspaceName

The default workspace (must exist in CloudBees CD server instance) for the resources created in the CloudBees CD server for the agent(s).

Default: null

zoneName

The zone (must exist in CloudBees CD server instance) for the resources created in the CloudBees CD server for the agent(s).

Default: null

Agent resource name templates

Generally, you set the name for the agent resource with the value agent.resourceName , but this only works when the agent deployment has only one replica. If there are multiple replicas, then each of them tries to register itself using the same resource name. As a result, only one replica from the entire deployment will be registered as a resource.

The resource name defaults to hostname , but other template values are available to fully qualify the resource:

  • Ordinary helm template values. For example: {{ .Release.Name }} , {{ .Release.Namespace }} , and so on.

  • In addition, two special template values are allowed :

    • {{ hostname }} –the actual resource hostname

    • {{ ordinalIndex }} –the serial replica index in the StatefulSet

For example:

resourceName: "myResource - {{ .Release.Name }} - {{ ordinalIndex }}"

Troubleshooting

How to deploy monitoring support

Use Grafana to visualize the Elasticsearch metrics. Issue the following commands to deploy it at you r site.

helm install --name prometheus stable/prometheus
helm install --name grafana stable/grafana
# Note that username is admin and password is the result of following command
kubectl get secret --namespace default grafana -o jsonpath="{.data.admin-password}" | \
   base64 --decode ; echo RandomPassword

How to deploy logging support

In general, each object in a Kubernetes cluster produces its own logs. And usually the user has their own mechanism in place to manage logs from different services in the cluster. Logs from CloudBees CD services and pods can be captured with standard log shipper tools. A sample configuration file for the FileBeat log shipper is provided here .

How to increase memory limits for CloudBees CD components

During periods of high work load, a server component could run out of memory if it asks for more memory than is allocated to the JVM. In order to increase the memory for a component, we have to allocate more memory to the component’s container. Then, depending on the component, the memory allocation for the component, itself, running in the container needs to be increased accordingly. See Cluster capacity for default container memory settings.

The following configurations can be used to change the memory allocation for each container and component.

Component Container memory limit Component memory setting Example

CloudBees CD server

server.resources.limits.memory

server.ecconfigure

server.ecconfigure: "--serverInitMemoryMB=4096 --serverMaxMemoryMB=4096"

CloudBees CD web server

web.resources.limits.memory

N/A

Repository server

repository.resources.limits.memory

repository.ecconfigure

ecconfigure: " --repositoryInitMemoryMB=256 --repositoryMaxMemoryMB=512"

DevOps Insight server

dois.resources.limits.memory

dois.esRam (heap size in MB for Elasticsearch)

dois.lsInitRam , dois.lsMaxRam (heap size in MB for Logstash)

Bound agent

boundAgent.resources.limits.memory

boundAgent.ecconfigure

ecconfigure: "--repositoryInitMemoryMB=256 --repositoryMaxMemoryMB=512"

Inject new memory limits using helm . Update your local values file (here it is called myvalues.yaml ) with the new values and issue the helm `upgrade ` command.

helm upgrade  --name  \
-f  --namespace  --timeout 10000

How to resolve volume note affinity conflicts

Sometimes pods can hang in the `pending ` stage with the following error:

x/y nodes are available: y node(s) had volume node affinity conflict.

This can happen when the availability zone for the persistent volume claim is different from the availability zone of the node on which the pod gets scheduled.

A cluster administrator can address this issue by specifying the WaitForFirstConsumer mode, which delays the binding and provisioning of a PersistentVolume ` until a pod using the `PersistentVolumeClaim is created. PersistentVolume s are selected or provisioned conforming to the topology that is specified by the pod’s scheduling constraints.

Data backups and disaster recovery

Kubernetes backup tools such as Heptio’s Velero (previously known as Ark) are used to backup and restore Kubernetes clusters. CloudBees CD services and pods deployed in the cluster can be backed using those standard backup tools. Application data maintained in persistent volumes needs to be backed up at the same time. Tools such as Velero support persistent volume backups.

Here is the list of volumes associated with the CloudBees CD pods:

Container Volumes

flow-server

efs-deployment, logback-config,init-scripts

flow-agent

flow-agent-workspace, logback-config

flow-devopsinsight

elasticsearch-data

flow-repository

efs-repository, logback-config

flow-web

efs-deployment